On Tue, 1 Oct 2013, Oscar del Rio wrote:
On 10/ 1/13 08:49 PM, Andrew Daviel wrote:
We are using UW imapd 2007 with MIX format mailboxes.
Our storage expert would really like to use an NFS-mounted volume on a
native ZFS appliance, saying it's much better for snapshots and
block-oriented backups than iSCSI.
Thanks Oscar for your answers
I have been looking at this again, as our storage guy said "Oh but Linux
supports the local_lock switch in NFS mount".
Yes it does - from Linux 2.6.37. But I found a patch to backport that to
2.6.18, which is coincidentally what I am running on CentOS 5.
So I've been playing. I ran mailutil to repeatedly append and prune a
mailbox on NFS, while dmail keeps delivering mail. Sure enough, after a
few minutes, I get mailbox corruptions.
Then I built a patched nfs_utils and nfs.ko module, and tried mounting the
disk with e.g. mount -t nfs -o rw,nosuid,local_lock=all
UW imap defines safe_flock in c-client/flocklnx.c, which makes flock()
a no-op, citing https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=123415
But that's from 2005, talking about Linux 2.4.21.
http://en.wikipedia.org/wiki/File_locking says that
"Kernel 2.6.12 and above implement flock calls on NFS files using POSIX
byte-range locks" and the kernel code fs/nfs/file.c says
"We're simulating flock() locks using posix locks on the server. No BSD
flocks over NFS allowed.' Seems to me flock() works now. The flock shell
command works fine on an NFS volume (flock lockfile bash). In spite of the
fact the manpage for flock() (2) still says it doesn't.
So I commented out the no-op in safe_flock, and my test setup works fine -
no mailbox corruption - even with local_lock=none. However, it does
occasionally hang for a bit - it may be triggering the 5-second sleep in
safe_flock() although I don't see any syslog messages. Or it may be the
NFS server itself.
If I set local_lock=all, again I get no corruption, and I don't get any
hangs either.
I haven't tried exhaustively yet - running the test for hours - or a
proper mailtest suite.
If you are planning to change storage, maybe it is also time to consider
switching to another server.
We recently moved to dovecot with mdbox format, which is similar to Mark's
MIX format (but not compatible).
I found e.g.
http://www.dovecot.org/list/dovecot/2012-June/066291.html
http://wiki.dovecot.org/NFS
which implies Dovecot may not be 100% robust on NFS.
--
Andrew Daviel, TRIUMF, Canada
Tel. +1 (604) 222-7376 (Pacific Time)
_______________________________________________
Imap-uw mailing list
Imap-uw@u.washington.edu
http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
