Here are my now somewhat out of date notes (updated from that
older .rtf version) for getting DSPAM going on OS X Server 10.3 (I
think we're at 10.4.6 now, but that shouldn't matter all that much).
I wouldn't bother with corpus training--just start using DSPAM. Also,
we ended up with a single global DSPAM group which seems to work well
enough for our little 10-user system.
Hope this helps. No warrantees, YMMV, etc.
- - -
How to install DSPAM ¶
(These are now a bit out of date; this was for OS X Server 10.3 and
DSPAM 3.2, and for the stock Apache 1.3. See the change log for
details on installing the latest DSPAM. So take this just as
background material.)
DSPAM is a statistical hybrid filter for SPAM. It is very similar in
nature to the junk mail filter that's built into Mail.app, but with
one major advantage: the message processing is done on your mail
server, not on the mail client.
By processing the messages on the server, you eliminate the need to
download hundreds of messages just to throw them into a junk mail
folder. Also, keeping statistics about SPAM on the server allows an
administrator to build composite information gathered from all users
of the mail server--allowing new users to benefit from the SPAM
caught by previous users.
The following instructions will help you configure DSPAM on a stock
Mac OS X Server installation (version 10.3.x). This configuration
utilizes the existing tools that are installed: Postfix, Cyrus.
(These instructions also don't use procmail at all, assuming that the
users are all remote IMAP/POP users and not shell users who'd be
fiddling with .forward, etc.)
To perform these steps, you should be feel comfortable using command
line tools to download, configure, build and install software.
The latest version of DSPAM is available at the following URL:
http://www.nuclearelephant.com/projects/dspam/
These instructions were written for version 3.2.6 and assume that you
have reviewed the README file from the DSPAM download. It is also
assumed that you've put the DSPAM source code in ~/dspam-3.x.y/ after
downloading. ("X" and "y" are minor and patch levels, of course, not
literally x and y.)
MySQL ¶
Make sure MySQL 4.1 or later is installed.
We assume MySQL is used as the backend storage for DSPAM, since it's
about the most efficient of the SQL back ends and also scales well.
DSPAM 3.x requires MySQL 4.1 or later for best operation, so install
it from mysql.com somewhere on your system--we assume it's installed
in /usr/local/mysql and below, so vary the instructions to match
where you install it.
(MySQL installation is not covered here, but it's pretty simple for
OS X these days.)
Set some basic MySQL tuning parameters in /etc/my.cnf (which you may
have to create--note that the [mysql] is part of the file):
[mysqld]
set-variable = max_allowed_packet=8192000
key_buffer_size = 64M
myisam_sort_buffer_size = 64M
Make sure there's a "root" user in MySQL with the appropriate
password, which we'll use later.
Download DSPAM ¶
where 3.2.6 is replaced by the current version number:
$ mkdir ~/dspam
$ cd ~/dspam
$ curl -O http://www.nuclearelephant.com/projects/dspam/sources/
dspam-3.2.6.tar.gz
$ tar xfz dspam-3.2.6.tar.gz
$ cd dspam-3.2.6
Patch MySQL Library ¶
Patch libmysqlclient.a because DSPAM build will fail without this patch:
$ cd /usr/local/mysql/lib
$ mv libmysqlclient.a libmysqlclient.a.original
$ mkdir /tmp/mysql
$ cd /tmp/mysql
$ ar x /usr/local/mysql/lib/libmysqlclient.a.original
$ ld -r -d my_error.o
$ mv a.out my_error.o
$ ld -r -d charset.o
$ mv a.out charset.o
$ cd /usr/local/mysql/lib
$ libtool -o libmysqlclient.a /tmp/mysql/*.o
(and ignore "no symbols" warnings).
More information on how this patch works can be found here: http://
curl.haxx.se/mail/lib-2002-06/0202.html .
Configure and Build DSPAM ¶
To configure DSPAM to run with MySQL as the backend, use something
like the following.
(Our site likes to put everything (including variable stuff) for a
locally-installed package like dspam under /usr/local/dspam, rather
than scatter binaries, libraries, include files, etc. around /usr/
local/bin, /usr/local/lib, /usr/local/include, /usr/local/var, etc.
The only drawback is that DSPAM's variable data ends up under /usr/
local/dspam/var/dspam, but that extra level isn't really a problem.)
$ cd ~/dspam/dspam-3.x.y
$ ./configure \
--prefix=/usr/local/dspam \
--exec_prefix=/usr/local/dspam \
--sysconfdir=/usr/local/dspam/etc \
--with-storage-driver=mysql_drv \
--with-mysql-includes=/usr/local/mysql/include \
--with-mysql-libraries=/usr/local/mysql/lib \
--enable-virtual-users \
--enable-debug
$ make
$ sudo make install
This will create and populate /usr/local/dspam/{lib,bin,man,etc,var/
dspam,...} and set up all file permissions properly.
If you have other sysadmins that need to run dspam from the command
line but not as root, then you should make dspam world-executable:
$ sudo chmod o+rx /usr/local/dspam/bin/dspam
and add Trust lines in the dspam.conf file for the users in question,
as detailed below.
Create MySQL databases for DSPAM ¶
$ /usr/local/mysql/bin/mysqladmin create dspam -u root -p
Enter password:
$ /usr/local/mysql/bin/mysql -u root -p dspam < tools.mysql_drv/
mysql_objects-4.1.sql
Enter password:
$ /usr/local/mysql/bin/mysql -u root -p dspam < tools.mysql_drv/
virtual_users.sql
Enter password:
Copy the nightly purging script and set it up for cron execution as
root (making sure to use our local mysql, not the stock OS X mysql):
$ sudo cp tools.mysql_drv/purge-4.1.sql /usr/local/dspam/var/dspam
$ echo > /tmp/purge "0 0 * * * /usr/local/mysql/bin/mysql -u root --
password={mysql root password} dspam < /usr/local/dspam/va\
r/dspam/purge-4.1.sql"
$ sudo crontab -u root /tmp/purge
Setup DSPAM's configuration files ¶
Edit /usr/local/dspam/etc/dspam.conf to change the MySQL
configuration lines to:
MySQLServer /tmp/mysql.sock
MySQLPort
MySQLUser root
MySQLPass "MySQL root user password here"
MySQLDb dspam
MySQLCompress false
Add the user ids that will need to run DSPAM to /usr/local/dspam/etc/
dspam.conf; you'll like only need to add the following Trust lines
after all the other Trust lines (cyrus here because we're going to
call dspam from Postfix as user cyrus, and www because we're going to
call dspam from the cgi machinery as user www):
Trust admin
Trust cyrus
Trust www
Trust <other sysadmin>
Make the trusted local delivery agent Cyrus' local delivery tool.
TrustedDeliveryAgent "/usr/bin/cyrus/bin/deliver"
Change the line
Preference "showFactors=on/off"
as you want the full token factors in each message header or not
(should probably be off).
Create and initialize a global user for merged DSPAM statistics ¶
If it makes sense for your site, you can create a global user for
merged DSPAM statistics with the steps in this section.
Create a "global" user with Workgroup Manager.
Create or edit the file /usr/local/dspam/var/dspam/group and make
sure the following line is present:
global:merged:*
Download the DSPAM SpamAssassin? corpus trainer and various corpora
from the SpamAssassin? corpus, as below. You'll have to adjust these
files names to match what's in the corpus, and we do this in two
steps (2002 and 2003) because otherwise the directories will clobber
each other. Also, we have to sudo the training because the dspam
executable needs to run as superuser.
$ PATH=$PATH:/usr/local/dspam/bin; export PATH
$ curl -O http://dspam.nuclearelephant.com/sources/extras/
dspam_sa_trainer.tar.gz
$ gunzip dspam_sa_trainer.tar.gz
$ cd dspam_sa_trainer
$ curl -O "http://spamassassin.apache.org/publiccorpus/20021010_
{easy_ham,hard_ham,spam}.tar.bz2"
...downloading messages...
$ bunzip2 *.bz2
$ for i in *.tar; do tar xf $i; done
$ rm *.tar
$ sudo perl publiccorpus.pl global
...searching/training messages...
$ curl -O "http://spamassassin.apache.org/publiccorpus/20030228_
{easy_ham,hard_ham,spam,spam_2}.tar.bz2"
...downloading messages...
$ bunzip2 *.bz2
$ for i in *.tar; do tar xf $i; done
$ rm *.tar
$ sudo perl publiccorpus.pl global
...searching/training messages...
$ sudo dspam_clean -p0 global
Check that global reporting aliases ham-global and spam-global are
in /etc/aliases (see below). These can be used by other users to
train the global user (by forwarding messages to the correct address).
Setup DSPAM CGI ¶
Copy the entire contents of ~/dspam/dspam-3.x.y/cgi folder
(recursively, to include the templates) to a newly-created folder /
Library/WebServer/Documents/DSPAM.
$ cd ~/dspam/dspam-3.x.y
$ cp -r cgi /Library/WebServer/Documents/
$ mv /Library/WebServer/Document/{cgi,DSPAM}
Change the owner of all CGI scripts (*.cgi) and Perl (*.pl) scripts
to `www, and make sure they're all writable and executable by group
(writable because you'll have to edit at least one of them):
$ cd /Library/WebServer/Documents/DSPAM
$ sudo chown www *
$ sudo chmod g+rwx *.cgi *.pl
Add any global CGI administration users you want to the admins file
(those who can administer other users and see overall system stuff).
Edit configure.pl and change the site-specific info appropriately,
after # Add customized settings below. At least change the
LOCAL_DOMAIN setting to match your hostname. Everything else should
be configured properly by your original DSPAM configuration.
You'll also need to set the DSPAM_ARGS to something matching your
dspam delivery method in master.cf.defaultserver. Ours is
$CONFIG{'DSPAM_ARGS'} = "--deliver=innocent --class=innocent " .
"--source=error --user $CURRENT_USER " .
"-m user/$CURRENT_USER -a $CURRENT_USER";
[but, in the 3.4RC1 release, we have to repeat this assignment
*after* CURRENT_USER is computed in dspam.cgi; this will be fixed].
Setup Graph::GD for graphics in Perl CGI scripts ¶
Download libpng from http://www.libpng.org/pub/png/libpng.html then
build and install:
$ cp scripts/makefile.darwin Makefile
$ make
$ sudo make install
(Ignore make errors for pngtest.)
Download libgd from http://www.boutell.com/gd, configure build and
install:
$ ./configure
$ make
$ sudo make install
$ sudo ranlib /usr/local/lib/libpng.a
Get GD::Graph from CPAN:
$ sudo cpan
cpan> install GD
installed? /usr/local/lib
support? [y to PNG only]
cpan> install GD::Graph::lines3d
cpan> quit
Using the Server Admin application, under Settings/Sites, create a
new web domain DSPAM enabled on port 8080 (or your favorite alternate
port). After setup, you will be accessing the DSPAM web tools with:
http://your.server.domain:8080/
- General: Set name to DSPAM, port to your desired port (8080 or what-
have-you). Set web folder to /Library/WebServer/Documents/DSPAM. Add
"dspam.cgi" to default index files (at top of list). Set the
administrator email to something sane.
- Options: Enable CGI Execution.
- Realms: Create a new realm named DSPAM, add users who will be
checking & processing spam with browse and author privileges, add
group "admin" with browse and author privileges. (You might also add
other groups that encompass most of your users--e.g., "dspamusers",
to avoid adding them all individually.) And you might want to have
separate access and error log files, e.g., dspam_access_log and
dspam_error_log for easier log checking.
Exit realm creation (press the back-arrow button on the upper right)
and then enable the DSPAM domain which now appears in the Settings/
Sites listing.
Using the Workgroup Manager, add the "www" user to "mail" group so
that Apache can send mail.
Setup Postfix to send messages via DSPAM ¶
Add to end of /etc/postfix/master.cf.defaultserver (localize this for
your DSPAM bin folder):
dspam unix - n n - - pipe
flags=hqu user=cyrus argv=/usr/local/dspam/bin/dspam --mode=teft --
deliver=innocent --feature=chained,noise --user ${user} -\
m user/${user} -a ${user} -r ${sender}
(Don't use "R" in the flags, or you'll get an extraneous `Return-
path: <[EMAIL PROTECTED]>` header. We supply our own to cyrus'
deliver with the -r flag above.)
Note that this .defaultserver file is what the OS X Server Admin
machinery works with each time you restart the server. Add to /etc/
postfix/main.cf:
mailbox_transport = dspam
default_privs = cyrus
and be careful to comment our the other definitions of
mailbox_transport--there's one for Cyrus at the original end of the
file.
(The idea here is that all local mailbox delivery will go through the
"dspam" transport, and DSPAM itself will deliver to local mailboxes
(cf. the dspam.conf TrustedDeliveryAgent setting) with the Cyrus
deliver utility, whenever it needs to deliver something to users
locally.)
After updating the Postfix configuration, shut down the mail service,
remove the silly .swap file that confuses things, and restart the
service.
$ sudo serveradmin stop mail
$ rm /etc/postfix/master.cf.defaultserver.swap
$ sudo serveradmin start mail
DSPAM uses special mail aliases, spam-'user' and ham-'user', that
update the per-user statistics. Create these aliases by editing the
aliases file (as reported by the "postconf alias_maps" command),
usually /etc/aliases:
cyrus: root
debug: root
spam-'user': "|/usr/local/dspam/bin/dspam --user 'user' --
class=spam --source=error --mode=teft"
ham-'user': "|/usr/local/dspam/bin/dspam --user 'user' --
class=innocent --source=error --mode=teft --deliver=innocent -m u\
ser/'user' -a 'user'"
spam-global: "|/usr/local/dspam/bin/dspam --user global --
class=spam --source=corpus --mode=teft"
ham-global: "|/usr/local/dspam/bin/dspam --user global --
class=innocent --source=corpus --mode=teft"
Replace 'user' with a real user name, so there is a spam-user and ham-
user for every user who receives mail on the server (note that we
have to supply the actual delivery information to dspam in the ham
case, since the message should be delivered). The spam-global and ham-
global are used to update the corpus used by all users.
The cyrus and debug aliases are used for debugging.
After the aliases are updated, deploy them with:
$ sudo newaliases
Test ¶
At this point, any mail sent to a user account on the server will
pass through DSPAM before being delivered. Any mail that is put into
the quarantine (because it is thought to be spam) can be viewed with
your web browser:
http://mail.yourserver:8080/
Of course, nothing ever goes as planned, so if you have problems here
are some good places to look:
/var/mail
for messages that are not getting delivered correctly
/var/log/mail.log
Postfix and Cyrus message processing
/var/log/system.log
Postfix and Cyrus errors
/usr/local/dspam/var/dspam/dspam.debug
DSPAM processing log (if enabled during configure)
/usr/local/dspam/var/dspam/dspam.messages
DSPAM message log (if enabled during configure)
/usr/local/dspam//var/dspam/sql.errors
DSPAM errors with MySQL backend
/usr/local/dspam/var/dspam/data/user/user.log
A log of messages processed for user (along with status)
/usr/local/dspam/var/dspam/data/user/user.mbox
The quarantine for a given user
/etc/postfix/master.cf.defaultserver
Postfix transport configurations
/etc/postfix/main.cf
Postfix configuration
/etc/imap.conf
Cyrus configuration
On Jan 24, 2007, at 10:52 PM, Gregory S. Combs wrote:
So I've got Mac OS X Server 10.4.8 which runs cyrus, postfix, spam
assassin, clamav, and a goofy GUI.
In order to keep the GUI running, it's inadvisable to recompile the
binaries used for the mail system. I don't particularly like the
way you train and manage spam with spamassassin under OSX, so I'm
very interested in getting DSPAM working here.
I've previously had great success using DSPAM with exim and dovecot
or cyrus on linux. However, I'd prefer to keep as much stock as
possible under OSX Server, so that means no exim. Additionally,
I've seen instructions from Richard5.net that use PCRE with
postfix, however it looks like PCRE isn't compiled into the postfix
for OSX Server. At least not in the same fashion that Richard5
does it.
I've seen probably 5 or 6 recipes for getting dspam on cyrus and
postfix, but for whatever reason I can't seem to get things going
right on this machine. There used to be an RTF floating around on
how to get dspam on OSX Server, but that's evidently disappeared.
What I'm looking for is a solution with minimal impact to the
current installation. I figure LMTP is best, due to the reduced
dependency on clients and permissions, but whatever works in the
easiest fashion is probably best.
If possible I'd like to retain the ability to forward spam and ham
to [EMAIL PROTECTED], or at the very least have a working
webgui ... I've seen solutions that maintain IMAP folders, but I
think this is going a little out of the way at this time.
Would any of you happen to have a recipe for getting dspam up on OS
X Server without having to recompile the standard mail system
binaries?
If not OS X "Server", how about a full recipe with postfix and
cyrus on OS X without using PCRE?
Thanks for any help you can send my way.
Greg
Cheers!
--Chris Ryland / Em Software, Inc. / www.emsoftware.com
