On 20.1.2012, at 4.27, Stan Hoeppner wrote: >> I spent months looking into NFS related issues. I read through Linux and >> FreeBSD kernel source codes to figure out if there's something I could do to >> avoid the problems I see. I sent some patches to try to improve things, >> which of course didn't get accepted (some alternative ways might have been, >> but it would have required much more work from my part). The mail_nfs_* >> settings are the result of what I found out. They don't fully work, so I >> gave up. > > Yeah, I recall some of your posts from that time, and your frustration. > If an NFS config option existed to simply turn off the NFS client > caching, would that resolve most/all of the remaining issues? Or is the > problem more complex than just the file caching? I ask as it would seem > creating such a Boolean NFS config option should be simple to implement. > If the devs could be convinced of the need for it.
It would work, but the performance would suck. >> There are several huge Dovecot+NFS setups. They use director. It works well >> enough (and with the recent fixes, I'd hope perfectly). > > Are any of these huge setups using mdbox? Or does it make a difference? I think they're all Maildirs currently, but it shouldn't make a difference. The index files are the ones most easily corrupted, so if they work then everything else should work just as well. In those director setups there have been no index corruption errors. > I.e. Indexes are indexes whether they be maildir or mdbox. Would > Director alone allow the OP to avoid the cache corruption issues > discussed in this thread? Or would there still be problems due to the > split LDA setup? By using LMTP proxying with director there wouldn't be any problems. Or using director for IMAP/POP3 and not using dovecot-lda for mail deliveries would work too. >>> In this case the OP has Netapp storage. Netapp units support both NFS >>> exports as well as iSCSI LUNs. If the OP could utilize iSCSI instead of >>> NFS, switching to GFS2 or OCFS, do you see these cluster filesystem as >>> preferable for mdbox? >> >> I don't have personal experience with cluster filesystems in recent years >> (other than glusterfs, which had some problems, but most(/all?) were fixed >> already or are available from their commercial support..). Based on what >> I've heard, I'm guessing they work better than random-access-NFS, but even >> if there are no actual corruption problems, it sounds like their performance >> isn't very good. > > So would an ideal long term solution to indexes in a cluster (NFS or > clusterFS) environment be something like Dovecot's own index metadata > broker daemon/lock manager that controls access to the files/indexes? > Either a distributed token based architecture, or maybe something > 'simple' such as a master node which all others send index updates to > with the master performing the actual writes to the files, similar to a > database architecture? The former likely being more difficult to > implement, the latter having potential scalability and SPOF issues. > > Or is the percentage of Dovecot cluster deployments so small that it's > difficult to justify the development investment for such a thing? I'm not sure if such daemons would be of much help. For best performance the user's mail access should be redirected to the same server in any case, and doing that solves all the other problems as well. I've a few other clustering plans besides a regular NFS based setup, but all of them rely on user normally being redirected to the same server (exception: split brain operation when mails are replicated to multiple data centers).
