Training Spambayes
==================

The training process is automated by a script called "train.sh".  Just
run it in the /usr/local/spambayes-corpus/ directory::

    $ sh train.sh

I periodically gather all the unsures together into a Unix mbox file
with a set of commands like this:

cd /var/spool/spambayes/unsure
rm -f u.mbox
for m in *.msg ; do
    cat $m >> u.mbox
    echo "" >> u.mbox
    rm $m
done

I've never bothered to capture this in a shell script.  It's just
retrieved from root's bash history.

I then scp the u.mbox file to my computer and use my favorite mailer
to step through the mailbox partitioning it into three sets: spam
(goes into s.mbox), ham (goes into h.mbox) and mail I don't care about
(such as delivery status mails) which are deleted.  I scp the
resulting [sh].mbox files back to this machine in /tmp and cat them to
the end of the relevant SpamBayes output files:

    cd /usr/local/spambayes-corpus
    cat /tmp/s.mbox >> spam.mbox.cull
    cat /tmp/h.mbox >> ham.mbox.cull

(Again, this is all retrieved from bash history.)

I then rename the .cull files and run the training script:

    mv spam.mbox.cull spam.mbox
    mv ham.mbox.cull ham.mbox
    sh train.sh

This script is slightly different than the one Neil originally wrote.
It uses a later version of the train-to-exhaustion tool which can cull
messages from the database which score correctly the first time they
are classified.  This keeps the database size down and restricts it to
mail messages which are pertinent to the type of mails seen without
overloading it with a similar class of messages which might appear in
a large burst (such as a new type of spam which is often going to
score as unsure until it's been trained on).