Maybe this would clear it up:
Test: Is the connecting MTA one I want to block?
Possible result: yes or no (there is no maybe in this test)
False positive: I incorrectly characterize an MTA that I don't want to block as a "yes".
It depends on what you are measuring. If you are measuring what percentage of E-mail that is not from an MTA that you want to block that gets caught (for example, if the test blocks all mail from 192.0.2.25, but it accidentally blocks an E-mail from 192.0.2.26), then that definition of false positive would be accurate.
If you are measuring the non-spams that get caught, then a non-spam coming from either 192.0.2.26 or 192.02.2.25 would count as a false positive.
Using the first measure, the test would (by definition) have 0 false positives, unless there is a bug in the test. So the claim of 0 false positives is meaningless -- it just means that the test works the way that it claims. It's about the same as saying "My test that blocks the IP 192.0.2.25 doesn't block any E-mail from IPs other than 192.0.2.25".
Using the second measure, the test would have a certain percentage of false positives, that would let users of the test know how much of their non-spam would get caught. This is a very useful number.
The problem is that virtually everyone in the anti-spam industry measures non-spams that get caught.
Note there is nothing about spam in there. Nothing about content. Nothing about consent. This type of test is different than a test that tries to determine if a message is spam and thus has a different definition of false positive.
Correct -- you understand it perfectly. :)
The problem is that there are others who see "zero false positives", and assume (correctly!) that it means "no non-spam will get caught." If you use the alternative definition of false positive, you must state that you are using that definition, and make it clear what information you are trying to get across (IE "My test works the way it should", not "My test will never capture legitimate E-mail").
-Scott
---
Declude JunkMail: The advanced anti-spam solution for IMail mailservers.
Declude Virus: Catches known viruses and is the leader in mailserver vulnerability detection.
Find out what you've been missing: Ask about our free 30-day evaluation.
--- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]
To Unsubscribe: http://www.ipswitch.com/support/mailing-lists.html List Archive: http://www.mail-archive.com/imail_forum%40list.ipswitch.com/ Knowledge Base/FAQ: http://www.ipswitch.com/support/IMail/
