https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6247
--- Comment #49 from Warren Togami <[email protected]> 2009-12-16 12:37:01 UTC --- Weekly Masscheck 20091216 ======================= # SUMMARY for threshold 5.0: # Correctly non-spam: 246432 99.33% # Correctly spam: 174605 93.82% # False positives: 1670 0.67% # False negatives: 11501 6.18% # TCR(l=50): 1.958990 SpamRecall: 93.820% SpamPrec: 99.053% Old Scores score HABEAS_ACCREDITED_COI 0 -8.0 0 -8.0 score HABEAS_ACCREDITED_SOI 0 -4.3 0 -4.3 score RCVD_IN_BSP_TRUSTED 0 -4.3 0 -4.3 score RCVD_IN_SSC_TRUSTED_COI 0 -3.7 0 -3.7 Weekly Masscheck 20091216, with HABEAS and BSP Disabled ================================================== # SUMMARY for threshold 5.0: # Correctly non-spam: 246431 99.33% # Correctly spam: 174698 93.87% # False positives: 1671 0.67% # False negatives: 11408 6.13% # TCR(l=50): 1.959877 SpamRecall: 93.870% SpamPrec: 99.053% Interestingly, we perform BETTER with the the whitelists turned off. This is indicative that spamassassin is well balanced and pretty safe against FP's even before the whitelists come into play. Weekly Masscheck 20091216, new RP rules =================================== New Scores score RCVD_IN_RP_CERTIFIED 0.0 -2.0 0.0 -2.0 score RCVD_IN_RP_SAFE 0.0 -3.0 0.0 -3.0 # SUMMARY for threshold 5.0: # Correctly non-spam: 246432 99.33% # Correctly spam: 174651 93.84% # False positives: 1670 0.67% # False negatives: 11455 6.16% # TCR(l=50): 1.959939 SpamRecall: 93.845% SpamPrec: 99.053% Enabling the new RP rules made things slightly worse again, about mid-way between disabled and the old rules. This is an effective score of -5 for CERTIFIED and -3 for SAFE. This is the scoreset that we have voted to include in spamassassin-3.3.0 where the cumulative score of CERTIFIED is -5. Weekly Masscheck 20091216, new RP rules, doubled up ============================================== New Scores score RCVD_IN_RP_CERTIFIED 0.0 -5.0 0.0 -5.0 score RCVD_IN_RP_SAFE 0.0 -3.0 0.0 -3.0 # SUMMARY for threshold 5.0: # Correctly non-spam: 246432 99.33% # Correctly spam: 174631 93.83% # False positives: 1670 0.67% # False negatives: 11475 6.17% # TCR(l=50): 1.959526 SpamRecall: 93.834% SpamPrec: 99.053% Just to satisfy your request, here is the same test with -5 for CERTIFIED. That is an effective score of -8 for CERTIFIED and -3 for SAFE. Results are not improved from the above. Weekly Masscheck 20091216, all DNS whitelists disabled ================================================= # SUMMARY for threshold 5.0: # Correctly non-spam: 246431 99.33% # Correctly spam: 174698 93.87% # False positives: 1671 0.67% # False negatives: 11408 6.13% # TCR(l=50): 1.959877 SpamRecall: 93.870% SpamPrec: 99.053% Out of curiosity, I did the same test with all RP and DNSWL whitelists disabled. The results improved even further. It seems that spamassassin is just fine without the whitelists in FP safety. I suspect however that the whitelists are helpful in pushing the total scores over the edge in order to trigger auto-learn. Given this analysis I suspect the lower whitelist scores in 3.3.0 are entirely appropriate. I am next attempting to repeat this analysis with the larger mcsnapshot masscheck logs instead of the most recent weekly masscheck. -- Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the assignee for the bug.
