On Mon, Jan 26, 2004 at 04:28:59PM +0100, Heikki Levanto wrote:
> Sorry, I have to bother whole list with one more test.
> If you can read this, the message passed through without being stopped,
> and I am close to solving the mystery.
Ok, I have solved the problem.
Most of you can stop reading here. Those that are overly curious, and
have some udnerstanding of the way computers work, can read below for an
explanation:
Long time ago, I set up a spam filter for the mailing list
(SpamAssassin). It worked fine for dropping the obvious
penis-enlargement pill adverts, nigerian scams, and all the other
rubbish that fills most mailboxes these days.
The say I had installed this, was to pipe all mail through spamassassin,
which added a few headers. In a good case only one, that says that the
mail was not spam. It looked something like this:
X-Spam-Status: No, hits=-5.3 required=4.0 tests=IN_REP_TO,ORIGINAL_MESSAGE
If the mail was spam, it would say something like
X-Spam-Status: Yes, hits=5.6 required=4.0
tests=MSG_ID_ADDED_BY_MTA_3,REMOVE_PAGE
So, I put a rule in the mailman filters that said
X-Spam-Status: Yes
means spam, and is not to be sent to the list.
So far so good. But SpamAssassin is a clever program, it tries to learn
what looks like spam and what looks like good messages. It is possible
to train it with selected samples, but even without, it builds wordlists
from those mails it can be pretty sure about. This is called bayesian
filtering. It can only work after SA has seen enough mails to have
something to compare to.
Unfortunately, when it has seen enough mails, it staerts to use a test
rule called BAYES_01 or BAYES_80, or something like that. And, again
very unfortunately, the filter matching rule tries to be pretty clever,
so it finds the X-Spam-Status header, extracts the value I am looking
for ('Yes') and finds it in the middle of BAYES. Ergo, all mail is
suspipcious, and blocked.
Once I had sorted out the problem, the solution was quite obvious: Use
the proper header line for filtering. The line
X-Spam-Flag: Yes
gets added to spam mails, and only them. And it does not have extra
data appended to it. This is the one thing I should have checked for
from the beginning.
Anyway, problem solved, and I hope the mailing list will function all
right in the future. Sorry for the inconvinience, and for this off-topic
message. Some of you had asked, and I suppose many more had wondered, so
I figured I had better explain what happened.
Now I deserve a beer. Unfortunately I don't have any home brew in the
office, it will have to wait until I get home.
-H
--
Heikki Levanto heikki at indexdata dot dk "In Murphy We Turst"
