I can answer this...
A little groundwork first.
Some tests fire with a positive weight when the message is spam and
don't fire when the message is not spam. Generally, when something
doesn't fire it produces a zero weight. Most tests are like this.
Some tests, like SpamCheck, produce a postiive or negative weight when
they think the message is spam or ham respectively.
Some tests produce a negative weight when they think the message is
ham and produce a zero otherwise.
With that said, you can interpret the OK/KO scores this way.
OK+ === Test said it's spam, final result was spam (this is good!)
OK- === Test said it's ham, final result was ham (this is good!)
KO+ === Test said it's ham, final result was spam (this is not good
and would cause false negatives!)
KO- === Test said it's spam, final result was ham (this is not good
and would cause false positives!)
One final thing about this --- the final result is not entirely
perfect. That is, it's not reviewed by a human. Rather it's the final
result that Declude came up with based on all of the tests.
As a result, tests which tend to be "ahead of the pack" might be
penalized. For example, there are times when Message Sniffer will have
an IP rule that is not yet in any of the DNSBLs. When that happens
Declude will say the message is ham while MsgSniffer says it's spam
(and the message in this case actually is). This can happen any time
a single test has a result ahead of the others.
That said, the number of times this actually happens in real life are
small enough that it doesn't matter. There are so many tests involved
that the final result from Declude is extremely accurate so on the
whole the weights you might gather from these statistics will be very
effective and the graphs you are looking at tend to be a very accurate
representation of the tests' true performance.
More below...
BB> We then need to realy avoind tests with lots of OK- and KO-, unless it is
BB> indented behaviour, right?
More precisely - you want to weight these closer to zero than your other tests.
KO- talks about tests that produce a positive weight.
KO+ talks about tests that produce a negative weight.
In both cases, the more red you see the closer to zero you want your
weights to be. The more green you see the "stronger" you want your
weight to be.
For example, take a normal spam test that you weight positively when
it fires.
Suppose you would normally give that test a weight of 50 on your
system. If you have a KO- value that is high - then you will want that
weight to be lower.
I am developing an automated tuning system based on this kind of
analysis. If you'd like to talk to me about the math involve then let
me know. There are a few ways to do it and I haven't quite settled the
question yet - but it looks very promising.
Suppose you say that you want at least two tests to fire before you
hold and your hold weight is 80. Then your target for each test will
be 40 (if the test is perfect).
Then you might calculate your weight for a test using the following
formula:
SS = [OK+]
SH = [KO-]
SpamWeight = 40*((SS-SH)/(SS+SH))
So, if your're talking about SNIFFER then you have an SS of 28021 and
an SH of 345. So you would set your weight:
SpamWeight = 40*((28021-345)/(28021+345))
= 40*(27676/28366)
= 40*(0.975675)
= 39.027
... ok, 39.
What about FIVETEN-SRC, (SS=20621, SH=3305)
SpamWeight = 40*(17316/23926)
= 40*(0.723731)
= 28.949
... ok, 29.
What about MTLDB, (SS=2509, SH=2317)
SpamWeight = 40*(192/4826) [ not looking good eh? ]
= 40*(0.039784)
= 1.591
... ok, 2
There are other more sensitive mathematical models for this, but this
is the simplest to explain.
BB> What about spamchk which has a lot of all results?
Most tests either fail or don't fail and the weight is set in Declude.
The spamchk test is different. It produces a variable weight for each
message and that weight might be positive (spammish) or negative
(hammish).
I'm not an expert on spamchk, but I believe you must do any tuning
in the .ini file since the result from spamchk is a variable number.
(If I have something wrong please let me know...)
Hope this helps,
_M
To Unsubscribe: http://www.ipswitch.com/support/mailing-lists.html
List Archive: http://www.mail-archive.com/imail_forum%40list.ipswitch.com/
Knowledge Base/FAQ: http://www.ipswitch.com/support/IMail/
