All, a few months back, there was a discussion on this list about the VBSpam comparative anti-spam tests[1], in which SpamAssassin performed significantly worse than many commercial products. Now I run these tests and I believe something was the matter with (the installation of) SA that made it perform so badly. For understandable reasons, none of the developers had time to help me set it up well for our test, so we decided to withdraw it for the time being.
I would still love to have the product back in the test. The test is paid-for, but free for free, open source products and we made that decision because we really wanted to have SA and others in the test. Now some people offered on this list to help me and that is why I'm writing this email -- Justin is happy for the community to help me. If there are people who are willing to help me set up SA so that it runs in ideal circumstances for our test, could they reply to me off-list[2] at this address or, even better, at martijn.groo...@virusbtn.com. A couple of things: - the main MTA for the test runs Qpsmtpd[3] on SUSE Linux Enterprise Server 11 and SA is run as a Qpsmtd-plugin; - from what is seems, all that SA was (and is) doing is doing some heuristic checks on the body of the email, which makes it catch about 50% of spam, with relatively many (several per cents) false positives; it checks every hour or so for updates, but these are rarely found; - I'm happy to add any extensions as long as these are also free and open source -- note that our 'target audience' includes big ISPs and unfortunately for them things as Spamhaus's RBL aren't free; - we don't white-list good senders (or blacklist bad ones) in any product, nor do we give 'feedback' to the products[4]; - I won't include SA in the test before the developers are happy with it being included: I know that some of the above rules might disproportionally disadvantage SA, so I would understand if they were to decide they wouldn't want it to be included. It is not in our intention to make SA look bad! Thanks. Martijn. [1] http://www.virusbtn.com/vbspam [2] but, because I hate people who post once and ask to be contacted off-list, I will keep checking the list too! [3] http://smtpd.develooper.com/ [4] we do give generic feedback to developers though: e.g. hey, you blocked a lot of newsletters, or you missed a lot of spam in Japanese. In the end of the day, the goal of our test is to make products better.
