Re: Scoring question
Ok this is an interesting followup. Since I *know* that the gr_domain.cf file is being read (I needed to change the '15' scores in there to 1.5 to have a shot at receiving mail from providers like Mailchimp and Constant Contact) I added the score HTML_FONT_LOW_CONTRAST 10 To *that* file, recompiled rules, and restarted spamd and voila, it is now scored at 10 rather the 0.1 And per other comments here a number of senders do the low contrast thing so what I *really* want are the ones where it is practically *no* contrast, is COLORx / (COLORx+1) LSB in one of the colors. (looking at source pages i see background #ff and text #fe for example. That suggests that local.cf is either not being read, or something comes along and sets the parameters after it sets them? I note that gr_domain comes lexically *before* local.cf so that is kind of confusing to me. --Chuck On Mon, Oct 8, 2018 at 11:19 AM John Hardin wrote: > On Mon, 8 Oct 2018, Chuck McManis wrote: > > > Hi, > > > > I restart spamd everytime I change local.cf and unfortunately John > changing > > the value as you've suggested and as I mentioned I had already tried, > > doesn't actually change the score it assigns. Still trying to figure this > > out. > > ...whoops, I didn't catch that, sorry... > > SA supports multiple config files, they are parsed in > sorted-by-filename-order. Are there multiple .cf files in your local > config folder and is the score for that rule being set in one that sorts > after the one you're editing? > > Also: if the score was zero it would be suppressed entirely. The score is > probably 0.001 (advisory). > > The base scores are: > > score HTML_FONT_LOW_CONTRAST 0.713 0.001 0.786 0.001 > > ...so depending on your config that score may be coming from base, and > your local changes *should* be overriding it. > > And - silly question - are you *sure* you're editing the *correct* local > config file? > > > > --Chuck > > > > On Mon, Oct 8, 2018 at 8:33 AM Kevin A. McGrail > wrote: > > > >> I think he might also need to restart spamd/amavisd/whatever to have the > >> local.cf change take place. > >> -- > >> Kevin A. McGrail > >> VP Fundraising, Apache Software Foundation > >> Chair Emeritus Apache SpamAssassin Project > >> https://www.linkedin.com/in/kmcgrail - 703.798.0171 > >> > >> > >> On Mon, Oct 8, 2018 at 11:18 AM John Hardin wrote: > >> > >>> On Mon, 8 Oct 2018, Chuck McManis wrote: > >>> > I have been trying to tune scores to achieve better matches with spam > >>> that > is getting through. And one test which shows up is > >>> HTML_FONT_LOW_CONTRAST > which is being scored for 0. Doing a scan of my incoming mail flow > this > >>> is > a huge signal, perhaps even a disqualifying one as I have yet to find > a > legitimate piece of email where this is true and it isn't spam. > > So in my local.cf file I added: > > score HTML_FONT_LOW_CONTRAST 10 > > To have it add 10 points to the spam score, recompiled rules and > >>> restarted > spamd. Yet it still gives it 0 points. > > 0.0 HTML_FONT_LOW_CONTRAST BODY: HTML font color similar or identical > to > background > > How do I change this value? > >>> > >>> In your local config file: > >>> > >>>score HTML_FONT_LOW_CONTRAST 1.000 # or whatever score you > prefer > >>> > >>> > >>> The reason it scores so low in the base ruleset is the S/O is *very* > low > >>> in the masscheck corpora - 0.114 - 4% spam hits vs. 31% ham hits. > >>> > >>> You might want to be careful if you intend to treat that as a poison > pill > >>> by itself... > >>> > >>> I'll take a look at whether its performance can be improved. > >>> > >>> -- > >>> John Hardin KA7OHZ > http://www.impsec.org/~jhardin/ > >>> jhar...@impsec.orgFALaholic #11174 pgpk -a > jhar...@impsec.org > >>> key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 > >>> --- > >>>Politicians never accuse you of "greed" for wanting other people's > >>>money, only for wanting to keep your own money.-- Joseph Sobran > >>> --- > >>> 557 days since the first commercial re-flight of an orbital booster > >>> (SpaceX) > >>> > >> > > > > -- > John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ > jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org > key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 > --- >The call to let 16-year-olds vote is a call to amplify the votes >of teachers' unions. If you think political indoctrination in the >schools is bad now, wait until it has the direct power to tip >election results. -- Robert Tracinski > -
Re: Scoring question
On Mon, 8 Oct 2018, Chuck McManis wrote: Hi, I restart spamd everytime I change local.cf and unfortunately John changing the value as you've suggested and as I mentioned I had already tried, doesn't actually change the score it assigns. Still trying to figure this out. ...whoops, I didn't catch that, sorry... SA supports multiple config files, they are parsed in sorted-by-filename-order. Are there multiple .cf files in your local config folder and is the score for that rule being set in one that sorts after the one you're editing? Also: if the score was zero it would be suppressed entirely. The score is probably 0.001 (advisory). The base scores are: score HTML_FONT_LOW_CONTRAST 0.713 0.001 0.786 0.001 ...so depending on your config that score may be coming from base, and your local changes *should* be overriding it. And - silly question - are you *sure* you're editing the *correct* local config file? --Chuck On Mon, Oct 8, 2018 at 8:33 AM Kevin A. McGrail wrote: I think he might also need to restart spamd/amavisd/whatever to have the local.cf change take place. -- Kevin A. McGrail VP Fundraising, Apache Software Foundation Chair Emeritus Apache SpamAssassin Project https://www.linkedin.com/in/kmcgrail - 703.798.0171 On Mon, Oct 8, 2018 at 11:18 AM John Hardin wrote: On Mon, 8 Oct 2018, Chuck McManis wrote: I have been trying to tune scores to achieve better matches with spam that is getting through. And one test which shows up is HTML_FONT_LOW_CONTRAST which is being scored for 0. Doing a scan of my incoming mail flow this is a huge signal, perhaps even a disqualifying one as I have yet to find a legitimate piece of email where this is true and it isn't spam. So in my local.cf file I added: score HTML_FONT_LOW_CONTRAST 10 To have it add 10 points to the spam score, recompiled rules and restarted spamd. Yet it still gives it 0 points. 0.0 HTML_FONT_LOW_CONTRAST BODY: HTML font color similar or identical to background How do I change this value? In your local config file: score HTML_FONT_LOW_CONTRAST 1.000 # or whatever score you prefer The reason it scores so low in the base ruleset is the S/O is *very* low in the masscheck corpora - 0.114 - 4% spam hits vs. 31% ham hits. You might want to be careful if you intend to treat that as a poison pill by itself... I'll take a look at whether its performance can be improved. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- Politicians never accuse you of "greed" for wanting other people's money, only for wanting to keep your own money.-- Joseph Sobran --- 557 days since the first commercial re-flight of an orbital booster (SpaceX) -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- The call to let 16-year-olds vote is a call to amplify the votes of teachers' unions. If you think political indoctrination in the schools is bad now, wait until it has the direct power to tip election results. -- Robert Tracinski --- 557 days since the first commercial re-flight of an orbital booster (SpaceX)
Re: Scoring question
On Mon, Oct 8, 2018 at 9:52 AM RW wrote:
> Are you sure that you are actually using spamd? You wouldn't be the
> first to run an unnecessary spamd instance while the actual
> classifications are done in a different daemon, such as amavisd, using
> the spamassassin libraries.
>
That is a fair question, I believe I am and I'll give you two different
ways that I can reason to that conclusion.
The first is that I'm running Postfix. My master.cf in Postfix has the
following chain:
smtp -> smtpd - o content_filter=spamfilter
spamfilter -> pipe /usr/local/bin/spamck -f ${sender} ${recipient}
/usr/local/spamck has in it:
cat | /usr/local/bin/spamc -u filter > checked file.
And spamc sends mail through spamd in order to check it.
Second set of reasoning.
When I first started using spamassassin I discovered that the gr_domain.cf
file was poison pilling the mail list vendors Constant Contact and
Mailchimp (among others). Since I have users that actually get mail they
want from those providers I went in and manually dialed back the scores.
That followed by sa-compile then restart, and the new scores show up in
the spam header and they stopped flagging a false positives.
That said, and it reminded me that I had also tried whitelisting some
domains in local.cf (being victims of this aggressive scoring on the
gr_domain.cf file part) and that whitelisting did *not* work. So I suppose
it is possible that my local.cf file isn't actually being used by spamd.
(which seems odd, but I'm not sure how to check that definitively)
--Chuck
Re: Scoring question
On Mon, 8 Oct 2018 09:11:17 -0700 Chuck McManis wrote: > Hi, > > I restart spamd everytime I change local.cf and unfortunately John > changing the value as you've suggested and as I mentioned I had > already tried, doesn't actually change the score it assigns. Still > trying to figure this out. Are you sure that you are actually using spamd? You wouldn't be the first to run an unnecessary spamd instance while the actual classifications are done in a different daemon, such as amavisd, using the spamassassin libraries.
Re: Scoring question
Hi, I restart spamd everytime I change local.cf and unfortunately John changing the value as you've suggested and as I mentioned I had already tried, doesn't actually change the score it assigns. Still trying to figure this out. --Chuck On Mon, Oct 8, 2018 at 8:33 AM Kevin A. McGrail wrote: > I think he might also need to restart spamd/amavisd/whatever to have the > local.cf change take place. > -- > Kevin A. McGrail > VP Fundraising, Apache Software Foundation > Chair Emeritus Apache SpamAssassin Project > https://www.linkedin.com/in/kmcgrail - 703.798.0171 > > > On Mon, Oct 8, 2018 at 11:18 AM John Hardin wrote: > >> On Mon, 8 Oct 2018, Chuck McManis wrote: >> >> > I have been trying to tune scores to achieve better matches with spam >> that >> > is getting through. And one test which shows up is >> HTML_FONT_LOW_CONTRAST >> > which is being scored for 0. Doing a scan of my incoming mail flow this >> is >> > a huge signal, perhaps even a disqualifying one as I have yet to find a >> > legitimate piece of email where this is true and it isn't spam. >> > >> > So in my local.cf file I added: >> > >> > score HTML_FONT_LOW_CONTRAST 10 >> > >> > To have it add 10 points to the spam score, recompiled rules and >> restarted >> > spamd. Yet it still gives it 0 points. >> > >> > 0.0 HTML_FONT_LOW_CONTRAST BODY: HTML font color similar or identical to >> >background >> > >> > How do I change this value? >> >> In your local config file: >> >>score HTML_FONT_LOW_CONTRAST 1.000 # or whatever score you prefer >> >> >> The reason it scores so low in the base ruleset is the S/O is *very* low >> in the masscheck corpora - 0.114 - 4% spam hits vs. 31% ham hits. >> >> You might want to be careful if you intend to treat that as a poison pill >> by itself... >> >> I'll take a look at whether its performance can be improved. >> >> -- >> John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ >> jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org >> key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 >> --- >>Politicians never accuse you of "greed" for wanting other people's >>money, only for wanting to keep your own money.-- Joseph Sobran >> --- >> 557 days since the first commercial re-flight of an orbital booster >> (SpaceX) >> >
Re: Scoring question
I think he might also need to restart spamd/amavisd/whatever to have the local.cf change take place. -- Kevin A. McGrail VP Fundraising, Apache Software Foundation Chair Emeritus Apache SpamAssassin Project https://www.linkedin.com/in/kmcgrail - 703.798.0171 On Mon, Oct 8, 2018 at 11:18 AM John Hardin wrote: > On Mon, 8 Oct 2018, Chuck McManis wrote: > > > I have been trying to tune scores to achieve better matches with spam > that > > is getting through. And one test which shows up is HTML_FONT_LOW_CONTRAST > > which is being scored for 0. Doing a scan of my incoming mail flow this > is > > a huge signal, perhaps even a disqualifying one as I have yet to find a > > legitimate piece of email where this is true and it isn't spam. > > > > So in my local.cf file I added: > > > > score HTML_FONT_LOW_CONTRAST 10 > > > > To have it add 10 points to the spam score, recompiled rules and > restarted > > spamd. Yet it still gives it 0 points. > > > > 0.0 HTML_FONT_LOW_CONTRAST BODY: HTML font color similar or identical to > >background > > > > How do I change this value? > > In your local config file: > >score HTML_FONT_LOW_CONTRAST 1.000 # or whatever score you prefer > > > The reason it scores so low in the base ruleset is the S/O is *very* low > in the masscheck corpora - 0.114 - 4% spam hits vs. 31% ham hits. > > You might want to be careful if you intend to treat that as a poison pill > by itself... > > I'll take a look at whether its performance can be improved. > > -- > John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ > jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org > key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 > --- >Politicians never accuse you of "greed" for wanting other people's >money, only for wanting to keep your own money.-- Joseph Sobran > --- > 557 days since the first commercial re-flight of an orbital booster > (SpaceX) >
Re: Scoring question
On Mon, 8 Oct 2018, Chuck McManis wrote: I have been trying to tune scores to achieve better matches with spam that is getting through. And one test which shows up is HTML_FONT_LOW_CONTRAST which is being scored for 0. Doing a scan of my incoming mail flow this is a huge signal, perhaps even a disqualifying one as I have yet to find a legitimate piece of email where this is true and it isn't spam. So in my local.cf file I added: score HTML_FONT_LOW_CONTRAST 10 To have it add 10 points to the spam score, recompiled rules and restarted spamd. Yet it still gives it 0 points. 0.0 HTML_FONT_LOW_CONTRAST BODY: HTML font color similar or identical to background How do I change this value? In your local config file: score HTML_FONT_LOW_CONTRAST 1.000 # or whatever score you prefer The reason it scores so low in the base ruleset is the S/O is *very* low in the masscheck corpora - 0.114 - 4% spam hits vs. 31% ham hits. You might want to be careful if you intend to treat that as a poison pill by itself... I'll take a look at whether its performance can be improved. -- John Hardin KA7OHZhttp://www.impsec.org/~jhardin/ jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 --- Politicians never accuse you of "greed" for wanting other people's money, only for wanting to keep your own money.-- Joseph Sobran --- 557 days since the first commercial re-flight of an orbital booster (SpaceX)
Re: Scoring question
Rick Zeman wrote: > Does this score: > > 0.001 BAYES_50Bayesian spam probability is 40 to 60% > > seem to be rather low for something with a 50% probablity of being spam? > No, as it has a 50% probability of being nonspam too. 50% is the "exactly undecided" mark.
Re: Scoring question
Rick Zeman schrieb: Does this score: 0.001 BAYES_50Bayesian spam probability is 40 to 60% seem to be rather low for something with a 50% probablity of being spam? SA 3.2.1 run within Maia with autolearning on. Tnx BAYES_50 means that bayes thinks that its 50% chance to be ham and 50% chance to be spam - so bayes should stay neutral because it has no "opinion" on this message. arni
Re: Scoring question
On 8/14/2007 3:49 PM, Rick Zeman wrote: Does this score: 0.001 BAYES_50Bayesian spam probability is 40 to 60% seem to be rather low for something with a 50% probablity of being spam? Anything higher would seem to be a little high for something with a 50% probability of being ham. Daryl
Re: scoring question
Miles Fidelman wrote: > Hi, > > I got the following in a message from our list management software: > > *X-Spam-Status: * Yes, hits=9.7 tagged_above=0.0 required=6.3 > tests=AWL, BAYES_20, NO_RELAYS > *X-Spam-Level: * * > *X-Spam-Flag: * YES > > Basic configuration: > Debian Sarge > Postfix > amavisd-new > spamassassin 3.001003 > standard ruleset, plus updates from > - default channel > - saupdates.openprotect.com > > The thing is, that if I'm reading things correctly, the scores for the > listed tests are: > AWL 1 (default) Nope... the AWL has a variable score. It's the "Automatic whitelist" which is really more of a "History-tracking score averager" than anything else. It's only called AWL because its most common effect is to push down scores when a normally low-scoring sender sends a message that gets a high score. In this case, it went the other way. A sender that was high-scoring in the past sent a low scoring message and got pushed up. > 50_scores.cf:score BAYES_20 0.0001 0.0001 -0.740 -0.740 > 50_scores.cf:score NO_RELAYS -0.001 > > Which should add up to .259 (net tests and Bayes turned on). > > So... why is this showing hits=9.7? What am I missing? See above, the variable score for the AWL would have been on the order of +9.45 or so. Apparently the past average for this sender is somewhere around +20, causing the AWL to add a lot to this message. The AWL score is based on the current pre-awl score, and the past average for that sender. Basically the AWL always looks at the difference between the current score, and the past average. It then adds half that difference in. See http://wiki.apache.org/spamassassin/AutoWhitelist > > > >
