David Severance wrote:
Mark Crispin wrote:
This is what I call an "attractive nuisance"; it is an easy-to-deploy
pseudo-solution, but is fundamentally wrong on an architectural level
and has severe technical shortcomings.
A great deal of the effort is in educating people as to why the
seemingly easy way is not the right way. Sadly, the lesson often
does not sink in until after the site has invested a considerable
amount of money in expensive hardware that does not solve their
problems.
This has been an interesting discussion for me and I have some further
comments. Here are the two systems I see as being advocated and why
each has severe shortcomings.
The NFS solution
Centralized storage (a NetApp in our case) which is shared via NFS to
a pool of IMAP servers behind a Foundry ServerIron doing a nice job of
load balancing users. Clients connect and are sent to whichever IMAP
server is least busy and each server accesses files via NFS. The
pluses for this approach are a robust storage system with alerts,
double parity and enterprise quality. Shared IMAP servers so no user
is ever down if a server breaks, the load balancer sends the user to
another machine and we are alerted. The load balancer itself can also
be configured to fail over to another load balancer (although the
budget for that hasn't materialized yet). It's also easy to configure
for end users who just use one FQDN to configure their software. The
minuses for this approach are that we are stuck using mbox due to NFS
lock issues and even then with limitations. We are happy to accept the
only one client can access the mail store and force that limitation
back to our users who live with it. The cooperative locking works just
fine for sendmail/procmail and IMAP. For 99.999% of our users this is
a non-issue. What isn't a non-issue is the performance difficulties
and the fact we limit our inbox size with quotas to manage this problem.
The Direct Attached method
What I have heard (or misunderstood from all the discussion) is that
Mark advocates lots of servers with direct storage. In this situation
I would probably take my NetApp and use either a SAN (if I had a
budget) or an iSCSI connection and carve out block storage to each
IMAP server. Each server would then host a group of end user accounts.
I would then have to either give different groups of end users
different FQDN's to attach to the service or use something like
Perdition (alternatives?) to route users from a single FQDN (like
imap.uci.edu) to the proper server. I might even use a load balancer
to front a couple of Perdition machines for redundancy. The pluses
would be that I could use far more efficient and scalable back end
mail formats (mix/mbx) and I would eliminate all of my performance
(fingers crossed) problems. It would eliminate that 1 in 10,000 users
who have a multiple client access issue. The minuses are that I need a
couple more machines (not the end of the world) and that a single IMAP
server failure will result in a service outage for some of my end
users. That's not acceptable and is a big problem for management as
well as it should be I might add. I also need to guess correctly and
distribute my users to various machines and balance them manually
instead of letting the load balancer do it dynamically and best of
all, without needing my intervention.
Thoughts
I'd like to change to something better for my 40,000 users but better
doesn't mean making the IMAP server a single point of failure for an
unlucky group of users. Surely there must be a better way to
accomplish this. I know there are various layers of SAN software which
can make a filesystem be shared but I've never used them and they are
expensive. It would seem to me that a better message store is
desirable. If the IMAP server ran off a database server I could use
the inherent clustering features of the database server and still keep
a pool of IMAP servers that could be shared among all my users. The
GFS file system that was briefly discussed earlier sounded interesting
but is also unproven in this environment. So, is there a solution now
that allows me to have a shared pool of IMAP servers which can access
whichever users files they need without NFS to avoid locking problems
or is the world imperfect and I should give up and use Exchange?
Please feel free to correct my misunderstandings and/or provide
(detailed) enlightenment. It's been an informative discussion so far :-)
David
The DBMail project ( http://www.dbmail.org/ ) is interesting and we
looked into it in detail to try to address this exact questio. However,
the last time I looked, 9-12 months ago, it was too immature and lacked
significant features compared to UW IMAP.
Presumably one could write a driver for UW IMAP that uses a MySQL (or
other) database and that could be clustered, etc.
-Erik Kangas
LuxSci.com
_______________________________________________
Imap-uw mailing list
[email protected]
https://mailman1.u.washington.edu/mailman/listinfo/imap-uw
