So if we were to insist on remote storage we would have to modify IMAP server source to do fcntrl instead of flocks where necessary and it still wouldn't work because lock files can't be counted on.
More importantly, you MUST use .lock files with NFS in addition to any system call locking.
I do not recommend fcntl() locking under any circumstances other than SystemV systems which do not have flock(). In such cases, "don't use NFS" changes from "good advice" to "mandatory".
It is mirrored on a separate server and is not on the same switch either.
That doesn't help if a virus gets onto that server and attacks its filesystem. The mirror will happily replicate the attack.
By breaking the user space into separate physical components with no interconnection, an attack must hit each of those components individually.
I want to understand this better. When you say each user account is on a separate piece of hardware I would take that to mean that you would then need a full system times 6 digits. Surely this isn't what you mean.
No. You are confusing virtual systems with hardware.
Each user has his own DNS name for his server. In my case, it is mrc.deskmail.washington.edu, and that is the only system that I connect to. Some number of users are on the same physical CPU. mrc.deskmail is currently on a machine called mailer52. mailer52 is also the ???.deskmail for other users.
New mailers are added as needed, and users get moved about to balance the load. At least one mailer is the doghouse for known resource-consuming users.
We haven't seen corrupted data, so I guess we've been lucky.
With traditional UNIX mailbox format, the most common form of corruption is lost messages. That's not something that you will notice easily.
Unless you mean to tell me that IMAP is never to be used in any type of file system besides a local file system and that is an inherent requirement / limitation of IMAP.
Given the state of the art today, services of the type that includes IMAP should not be used in any type of filesystem besides a local file system.
Certain higher-performance mail stores, such as the mbx format in UW IMAP and the format used by Cyrus IMAP, require a local filesystem and MUST NOT be used with NFS.
The reason for this is that there is more than synchronizing locks. It is also necessary to synchronize inode status and data. It is a requirement that a change anywhere in the file be instantaneously (atomicly) reflected in all other members in the cluster. This requires an *extremely* complex series of semaphores. As far as I know CFS on TOPS-20 is the *only* implementation to have gotten it right (although VMS clusters may also have done so).
Such semaphores require a huge amount of overhead. NFS and similar facilities punt on doing this entirely, based upon the assumption that most forms of file clustering don't need that level of service. It's the 99% solution that requires 1% of the work; and for that last 1% just say "don't use NFS for it" instead of putting in that extra 99% of work.
Databases (and mailboxes are a type of database) are part of the 1% that require 99% of the work.
-- Mark --
http://staff.washington.edu/mrc Science does not emerge from voting, party politics, or public debate. Si vis pacem, para bellum.
