On Mon, 27 Dec 1999, Michael Boman wrote:
> I am writing to you all looking for some answers for a future
> project.. Is it possible to do a cluster of qmail servers using DNS
> and NFS?
yes, although i'm not a big fan of nfs unless it's behind a STRONG
firewall. i set up something similar at the isp where i used to work, to
distribute the load among several mail servers. the system is still
running just as i set it up, although they've re-done one of the machines
(changed it from suse to redhat to match all of the others.) it does not
use nfs at all.
the trick is that we had several "mailhub" machines, with separate servers
for pop3 and virtual domains.
the publicly visible MX records all point to the mailhubs only, and the
/var/qmail/control/smtproutes file on each mailhub points each domain to
the correct machine (pop3 or virtual). this forces all incoming mail to
traverse one of these three machines, but removes that load from the pop3
server entirely.
i wrote a set of perl scripts that used ssh to automatically update the
config files on the various servers so that everything was in sync with
each other. it took a while to put together but once i finished it, it
worked really well (and is still working, even though i'm not there
anymore.)
the MX records for all domains point to "smtp1.blah.net",
"smtp2.blah.net", and "smtp3.blah.net" with non-equal weights. the trick
which distributes the load is that there are multiple A records.
the zone file for the primary domain (here called "blah.net") looks like
this:
smtp1.blah.net. IN A 10.0.0.5
smtp1.blah.net. IN A 10.0.0.6
smtp1.blah.net. IN A 10.0.0.7
smtp2.blah.net. IN A 10.0.0.6
smtp2.blah.net. IN A 10.0.0.7
smtp2.blah.net. IN A 10.0.0.5
smtp3.blah.net. IN A 10.0.0.7
smtp3.blah.net. IN A 10.0.0.5
smtp3.blah.net. IN A 10.0.0.6
blah.net. IN MX 1 smtp1.blah.net.
blah.net. IN MX 2 smtp2.blah.net.
blah.net. IN MX 3 smtp3.blah.net.
smtp.blah.net. IN CNAME smtp1.blah.net.
pop3.blah.net. IN A 10.0.0.9
virtual.blah.net. IN A 10.0.0.13
client domains are configured thusly...
client.com IN MX 1 smtp1.blah.net.
client.com IN MX 2 smtp2.blah.net.
client.com IN MX 3 smtp3.blah.net.
and the "smtproutes" files on the mailhubs looks like this:
blah.net:pop3.blah.net
client.com:virtual.blah.net
client2.com:virtual.blah.net
client3.com:virtual.blah.net
note again that this setup does not need nfs at all. the only need for nfs
would be if you had multiple machines doing pop3 duty, and nfs would
almost be wasted because the machine which physically housed the mailbox
directories would still be a single point of failure.
note also that all machines other than the mail servers (dns servers, web
servers, etc...) are running "mini-qmail" as detailed on one of djb's
pages, and the mailhubs are all running "qmail-qmqpd" under "tcpserver"
with the appropriate access control in order to handle outgoing mail.
the mailhubs should also be running "named", configured to only answer
queries from the localhost interface (i.e. they only serve themselves.) if
your site uses multiple name servers which update each other, the name
servers on the mailhubs should be updated with the name servers' normal
update cycle. this will save the network traffic from DNS queries and will
speed up qmail, since most DNS queries will be answered within the same
machine, and usually from named's cache.
the mailhubs are running qmail 1.03 with some anti-spam patches. they
don't need to be powerhouses, the ones i set up ranged from a pentium 166
to a pentium 350, each with 128MB RAM and 2-4GB disk each, running redhat.
the pop3 server is running qmail 1.03 with djb's "checkpassword" and the
imap server that comes as part of pine, with the bloodhound maildir
patches applied. the tcp-access list for the smtp/qmtp servers only allows
the mailhubs to send mail into the machine.
sqwebmail with system passwords could probably be installed here, but one
of the other guys there wrote a web-based mail reader in php that uses
imap on the local machine, and he's the one who now maintains the machine
so that's what is running. (i didn't find out about the inter7 programs
until after i left.)
the machine should have fast disks with enough space to store your users'
mailboxes, lots of memory, and enough CPU to drive it all. ours was a
sparc 2000e with 6 cpu's, 1GB RAM, and a 9GB scsi ultra-wide hard drive on
a dedicated differential controller for nothing but mailboxes, running
solaris because linux didn't quite work on the multi-processor sparc
machines when we got it.
the virtual domain server is running a custom-written interface that i did
long ago in perl and c, although the inter7 programs (vpopmail,
qmailadmin, sqwebmail, etc.) could be used here just as easily, and
probably would have smaller cpu and memory loads since the perl
interpreter wouldn't be used. it also only allows incoming messages
directly from the mailhubs.
this machine (when i left) was a pentium 300 with 128MB RAM and 9GB HD, i
hope they've upgraded it since then because the perl stuff, while easy to
write, was hideously slow.
to summarize...
- keep the mailhubs separate from the pop3 server
- make sure the mailhubs are doing nothing but qmail and named
- make sure the named's on the mailhubs are not answering queries from
other machines
- don't allow any machines other than the mailhubs to connect to the smtp
and/or qmtp ports on the pop3 and virtual machines
- run qmail-qmqpd on the mailhubs and set up all other internal servers
(dns servers, web servers, etc.) to run "mini-qmail"
----------------------------------------------------------------
| John Simpson | http://www.depeche.mode.net/~jms1/ |
| Programmer At Large | <[EMAIL PROTECTED]> |
----------------------------------------------------------------
| It all seems so stupid, it makes me want to give up. |
| But why should I give up when it all seems so stupid? |
----------------------------------------------------------------
