Lately, running SA 3.0.0 with no rule or score tuning, I have been
noticing that my false negatives tend to have BAYES_99 matched.
The scores file lists the following scores for Bayes:
50_scores.cf:score BAYES_00 0 0 -1.665 -2.599
50_scores.cf:score BAYES_05 0 0 -0.925 -0.413
50_scores.cf:score BAYES_20 0 0 -0.730 -1.951
50_scores.cf:score BAYES_40 0 0 -0.276 -1.096
50_scores.cf:score BAYES_50 0 0 1.567 0.001
50_scores.cf:score BAYES_60 0 0 3.515 0.372
50_scores.cf:score BAYES_80 0 0 3.608 2.087
50_scores.cf:score BAYES_95 0 0 3.514 2.063
50_scores.cf:score BAYES_99 0 0 4.070 1.886
I realize that these scores come out of the automated algorithm,
but they are not sensible on their face, and suggest a potential
problem with the Bayesian classifier's operation or the mass
check.
Note that even without network tests, BAYES_95 < BAYES_80, BAYES_60
With network tests, BAYES_05 is > BAYES_20, BAYES_40, and
BAYES_99 < BAYES_95 < BAYES_80.
It would not be unreasonable to constraint the BAYES_* scores
so that they are always monotonic in the predicted probability of
spam. This constraint would likely cause the scores associated with
other rules to change slightly, but might not reduce the overall
accuracy of SA in the mass check corpus (perhaps you're in some
kind of local minimum?)
I hope this makes sense. I'd be very interested in hearing about
other experiences with this.
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Alan Schwartz <[EMAIL PROTECTED]>
Author/Co-author of: "Managing Mailing Lists", "SpamAssassin",
"Stopping Spam", and "Practical Unix & Internet Security, 3rd Ed"
Published by O'Reilly Media, Inc. (http://www.oreilly.com)
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
