Re: [Dovecot] NFS issues
On Tue, Mar 23, 2010 at 03:19:49PM +0200, Timo Sirainen wrote: I have done some small-scale testing and it looks fine. Stress testing by running imaptest for same user's same mailbox in 2+ different servers (i.e. two NFS clients reading/writing same mailbox files) should show up quickly what kind of errors you could get. http://imapwiki.org/ImapTest OK, I've now set this up: ImapTest --- dovecot (same host) - NFS server `--- dovecot (diff host) ' * 172.16.23.104: dovecot 1.2.11 and ImapTest-latest. FreeBSD 7.2. * 172.16.23.101: dovecot 1.2.11 only. FreeBSD 7.2. * 172.16.23.103: NFS server. Ubuntu Karmic. All three hosts are ntpd synced. The following was needed on the FreeBSD boxes to get fcntl locking working: nfs_client_enable=YES rpc_lockd_enable=YES rpc_statd_enable=YES (imapd worked without these, but maillog showed errors about failing to obtain locks, operation not supported) Test results * Pointing a single instance of imaptest at a single host, or two instances of imaptest at the same host (with clients=5 to avoid hitting the 15 client limit) was fine. ImapTest reported no errors, and nothing out of the ordinary in maillog. $ egrep -v Login:|Disconnected:|Aborted login /var/log/maillog * Things went badly wrong with two instances of imaptest pointing at different dovecot hosts. I had seen this sort of thing when I'd previously been using dot locking, and was hoping they'd be fixed by switching to fcntl, but unfortunately not. ImapTest reported errors including: Error: br...@dev.example.com[8]: SELECT failed: 8.3 NO [SERVERBUG] Internal error occurred. Refer to server log for more information. [2010-03-25 10:22:23] - 6 stalled for 16 secs in command: 11 EXPUNGE All sorts of errors reported in maillog, including: Mar 25 10:22:23 freebsd-dev dovecot: IMAP(br...@dev.example.com): fscking index file /mail/0/6/37/30/brian%dev.example.com/dovecot.index Mar 25 10:22:23 freebsd-dev dovecot: IMAP(br...@dev.example.com): Transaction log /mail/0/6/37/30/brian%dev.example.com/dovecot.index.log: duplicate transaction log sequence (10) Mar 25 10:22:23 freebsd-dev dovecot: IMAP(br...@dev.example.com): Our dotlock file /mail/0/6/37/30/brian%dev.example.com/dovecot-uidlist.lock was overridden (locked 0 secs ago, touched 0 secs ago) Mar 25 10:22:23 freebsd-dev dovecot: IMAP(br...@dev.example.com): fscking index file /mail/0/6/37/30/brian%dev.example.com/dovecot.index Mar 25 10:22:23 freebsd-dev dovecot: IMAP(br...@dev.example.com): Transaction log /mail/0/6/37/30/brian%dev.example.com/dovecot.index.log: duplicate transaction log sequence (11) Mar 25 10:22:27 freebsd-dev dovecot: IMAP(br...@dev.example.com): /mail/0/6/37/30/brian%dev.example.com/dovecot.index reset, view is now inconsistent Mar 25 10:22:46 freebsd-dev dovecot: IMAP(br...@dev.example.com): Panic: file mail-transaction-log-view.c: line 108 (mail_transaction_log_view_set): assertion failed: (min_file_seq = max_file_seq) Mar 25 10:22:48 freebsd-dev dovecot: IMAP(br...@dev.example.com): rename(/mail/0/6/37/30/brian%dev.example.com/dovecot-uidlist.tmp, /mail/0/6/37/30/brian%dev.example.com/dovecot-uidlist) failed: No such file or directory Mar 25 10:22:48 freebsd-dev dovecot: IMAP(br...@dev.example.com): unlink(/mail/0/6/37/30/brian%dev.example.com/dovecot-uidlist.tmp) failed: No such file or directory Mar 25 10:22:36 wipe-dev dovecot: IMAP(br...@dev.example.com): ftruncate(/mail/0/6/37/30/brian%dev.example.com/dovecot-uidlist.lock) failed: Stale NFS file handle (Logs from a single test run are attached) Interestingly, these messages imply that dovecot is still using dotlocking in some circumstances, even though I've definitely set fcntl locking. $ grep ^lock /usr/local/etc/dovecot.conf lock_method = fcntl $ egrep '^mail_nfs|^mmap' /usr/local/etc/dovecot.conf mmap_disable = yes mail_nfs_storage = yes mail_nfs_index = yes All this suggests I should use some sort of 'sticky' load balancing in front so that all client conns from one IP hit the same frontend box. However, that contradicts the experience Adam McDougall has had with a similar setup: http://dovecot.org/list/dovecot/2010-March/047815.html It's possible that switching the Linux NFS server to a Netapp will help (which is what it will be deployed onto eventually anyway) Adam: did you do any tuning of FreeBSD client NFS settings? And have you tried using ImapTest, or just real IMAP users? I see there are a few tunables: $ grep nfs /etc/defaults/rc.conf netfs_types=nfs:NFS nfs4:NFS4 smbfs:SMB portalfs:PORTAL nwfs:NWFS # Net filesystems. nfs_client_enable=NO # This host is an NFS client (or NO). nfs_access_cache=60 # Client cache timeout in seconds nfs_server_enable=NO # This host is an NFS server (or NO). nfs_server_flags=-u -t -n 4 # Flags to nfsd (if enabled). nfs_reserved_port_only=NO # Provide NFS only on secure port (or NO). nfs_bufpackets= #
Re: [Dovecot] NFS issues
On Thu, 2010-03-25 at 10:58 +, Brian Candler wrote: Mar 25 10:22:23 freebsd-dev dovecot: IMAP(br...@dev.example.com): Our dotlock file /mail/0/6/37/30/brian%dev.example.com/dovecot-uidlist.lock was overridden (locked 0 secs ago, touched 0 secs ago) Wonder if dotlock_use_excl=no helps with this. Interestingly, these messages imply that dovecot is still using dotlocking in some circumstances, even though I've definitely set fcntl locking. Yes, dotlocking is always used for some files. It's possible that switching the Linux NFS server to a Netapp will help (which is what it will be deployed onto eventually anyway) NetApp should help, but I doubt it'll remove all the problems. Also Dovecot's NFS workarounds work better for Linux NFS client than for FreeBSD.. Maybe the load pattern from 'real' IMAP clients is such that these problems generally don't show in practice? (i.e. it would be unusual for a single IMAP client to make simultaneous changes to the same folder via different TCP connections) Right. They probably don't show up all that often for real users, but imaptest shows the worst case situation. signature.asc Description: This is a digitally signed message part
Re: [Dovecot] NFS issues
BTW. the triplicate send was caused by me breaking mailman's permissions for a while. Hopefully there won't be a fourth mail. :) signature.asc Description: This is a digitally signed message part
Re: [Dovecot] NFS issues
It's possible that switching the Linux NFS server to a Netapp will help (which is what it will be deployed onto eventually anyway) NetApp should help, but I doubt it'll remove all the problems. Also Dovecot's NFS workarounds work better for Linux NFS client than for FreeBSD.. I doubt this is related to the specific problem, but I've had issues with FreeBSD clients when using NFS over UDP. You might try forcing tcp from them: mount -o tcp host:/mount /mountpoint Rick
Re: [Dovecot] NFS issues
On Thu, Mar 25, 2010 at 04:31:56PM +0200, Timo Sirainen wrote: Mar 25 10:22:23 freebsd-dev dovecot: IMAP(br...@dev.example.com): Our dotlock file /mail/0/6/37/30/brian%dev.example.com/dovecot-uidlist.lock was overridden (locked 0 secs ago, touched 0 secs ago) Wonder if dotlock_use_excl=no helps with this. Doesn't appear to make a difference. grepping for 'dotlock': Mar 25 15:07:40 wipe-dev dovecot: IMAP(br...@dev.example.com): dotlock /mail/0/6/37/30/brian%dev.example.com/dovecot-uidlist.lock was immediately recreated under us Mar 25 15:07:43 wipe-dev dovecot: IMAP(br...@dev.example.com): dotlock /mail/0/6/37/30/brian%dev.example.com/dovecot-uidlist.lock was immediately recreated under us Mar 25 15:07:44 wipe-dev dovecot: IMAP(br...@dev.example.com): file_dotlock_create(/mail/0/6/37/30/brian%dev.example.com/dovecot-uidlist) failed: Input/output error Mar 25 15:07:45 wipe-dev dovecot: IMAP(br...@dev.example.com): dotlock /mail/0/6/37/30/brian%dev.example.com/dovecot-uidlist.lock was immediately recreated under us Mar 25 15:07:50 wipe-dev dovecot: IMAP(br...@dev.example.com): Our dotlock file /mail/0/6/37/30/brian%dev.example.com/dovecot-uidlist.lock was deleted (locked 1 secs ago, touched 1 secs ago) Mar 25 15:07:50 wipe-dev dovecot: IMAP(br...@dev.example.com): Our dotlock file /mail/0/6/37/30/brian%dev.example.com/dovecot-uidlist.lock was deleted (kept it 0 secs) Mar 25 15:07:57 wipe-dev dovecot: IMAP(br...@dev.example.com): Our dotlock file /mail/0/6/37/30/brian%dev.example.com/dovecot-uidlist.lock was deleted (locked 0 secs ago, touched 0 secs ago) Mar 25 15:08:06 wipe-dev dovecot: IMAP(br...@dev.example.com): dotlock /mail/0/6/37/30/brian%dev.example.com/dovecot-uidlist.lock was immediately recreated under us Mar 25 15:08:07 wipe-dev dovecot: IMAP(br...@dev.example.com): Our dotlock file /mail/0/6/37/30/brian%dev.example.com/dovecot-uidlist.lock was overridden (locked 1 secs ago, touched 1 secs ago) Mar 25 15:08:22 wipe-dev dovecot: IMAP(br...@dev.example.com): dotlock /mail/0/6/37/30/brian%dev.example.com/dovecot-uidlist.lock was immediately recreated under us Mar 25 15:08:26 wipe-dev dovecot: IMAP(br...@dev.example.com): Our dotlock file /mail/0/6/37/30/brian%dev.example.com/dovecot-uidlist.lock was deleted (kept it 0 secs) Mar 25 15:08:28 wipe-dev dovecot: IMAP(br...@dev.example.com): dotlock /mail/0/6/37/30/brian%dev.example.com/dovecot-uidlist.lock was immediately recreated under us Mar 25 15:08:34 wipe-dev dovecot: IMAP(br...@dev.example.com): Our dotlock file /mail/0/6/37/30/brian%dev.example.com/dovecot-uidlist.lock was deleted (kept it 0 secs) Mar 25 15:08:36 wipe-dev dovecot: IMAP(br...@dev.example.com): Our dotlock file /mail/0/6/37/30/brian%dev.example.com/dovecot-uidlist.lock was overridden (locked 0 secs ago, touched 0 secs ago) (lots of other errors also logged). And I'm still seeing ImapTest errors like Error: br...@dev.example.com[8]: STORE failed: 8.5 NO [SERVERBUG] Internal error occurred. Refer to server log for more information. [2010-03-25 15:07:44] Error: br...@dev.example.com[8]: DELETE failed: 8.6 NO [SERVERBUG] Internal error occurred. Refer to server log for more information. [2010-03-25 15:07:44] Error: br...@dev.example.com[8]: APPEND failed: 8.8 NO [SERVERBUG] Internal error occurred. Refer to server log for more information. [2010-03-25 15:07:45] 31121100114 5/ 5 Error: br...@dev.example.com[10]: Unexpected BYE: * BYE IMAP session state is inconsistent, please relogin. Regards, Brian.
Re: [Dovecot] NFS issues
On Thu, 2010-03-25 at 15:14 +, Brian Candler wrote: On Thu, Mar 25, 2010 at 04:31:56PM +0200, Timo Sirainen wrote: Mar 25 10:22:23 freebsd-dev dovecot: IMAP(br...@dev.example.com): Our dotlock file /mail/0/6/37/30/brian%dev.example.com/dovecot-uidlist.lock was overridden (locked 0 secs ago, touched 0 secs ago) Wonder if dotlock_use_excl=no helps with this. Doesn't appear to make a difference. grepping for 'dotlock': Are you using ext4 on the Linux NFS server? Nano/microsecond resolution timestamps fix the worst caching problems. signature.asc Description: This is a digitally signed message part
Re: [Dovecot] NFS issues
On 03/25/10 06:58, Brian Candler wrote: On Tue, Mar 23, 2010 at 03:19:49PM +0200, Timo Sirainen wrote: I have done some small-scale testing and it looks fine. Stress testing by running imaptest for same user's same mailbox in 2+ different servers (i.e. two NFS clients reading/writing same mailbox files) should show up quickly what kind of errors you could get. http://imapwiki.org/ImapTest OK, I've now set this up: ImapTest --- dovecot (same host) - NFS server `--- dovecot (diff host) ' * 172.16.23.104: dovecot 1.2.11 and ImapTest-latest. FreeBSD 7.2. * 172.16.23.101: dovecot 1.2.11 only. FreeBSD 7.2. * 172.16.23.103: NFS server. Ubuntu Karmic. All three hosts are ntpd synced. The following was needed on the FreeBSD boxes to get fcntl locking working: nfs_client_enable=YES rpc_lockd_enable=YES rpc_statd_enable=YES (imapd worked without these, but maillog showed errors about failing to obtain locks, operation not supported) Test results * Pointing a single instance of imaptest at a single host, or two instances of imaptest at the same host (with clients=5 to avoid hitting the 15 client limit) was fine. ImapTest reported no errors, and nothing out of the ordinary in maillog. $ egrep -v Login:|Disconnected:|Aborted login /var/log/maillog * Things went badly wrong with two instances of imaptest pointing at different dovecot hosts. I had seen this sort of thing when I'd previously been using dot locking, and was hoping they'd be fixed by switching to fcntl, but unfortunately not. ImapTest reported errors including: Error: br...@dev.example.com[8]: SELECT failed: 8.3 NO [SERVERBUG] Internal error occurred. Refer to server log for more information. [2010-03-25 10:22:23] - 6 stalled for 16 secs in command: 11 EXPUNGE All sorts of errors reported in maillog, including: Mar 25 10:22:23 freebsd-dev dovecot: IMAP(br...@dev.example.com): fscking index file /mail/0/6/37/30/brian%dev.example.com/dovecot.index Mar 25 10:22:23 freebsd-dev dovecot: IMAP(br...@dev.example.com): Transaction log /mail/0/6/37/30/brian%dev.example.com/dovecot.index.log: duplicate transaction log sequence (10) Mar 25 10:22:23 freebsd-dev dovecot: IMAP(br...@dev.example.com): Our dotlock file /mail/0/6/37/30/brian%dev.example.com/dovecot-uidlist.lock was overridden (locked 0 secs ago, touched 0 secs ago) Mar 25 10:22:23 freebsd-dev dovecot: IMAP(br...@dev.example.com): fscking index file /mail/0/6/37/30/brian%dev.example.com/dovecot.index Mar 25 10:22:23 freebsd-dev dovecot: IMAP(br...@dev.example.com): Transaction log /mail/0/6/37/30/brian%dev.example.com/dovecot.index.log: duplicate transaction log sequence (11) Mar 25 10:22:27 freebsd-dev dovecot: IMAP(br...@dev.example.com): /mail/0/6/37/30/brian%dev.example.com/dovecot.index reset, view is now inconsistent Mar 25 10:22:46 freebsd-dev dovecot: IMAP(br...@dev.example.com): Panic: file mail-transaction-log-view.c: line 108 (mail_transaction_log_view_set): assertion failed: (min_file_seq= max_file_seq) Mar 25 10:22:48 freebsd-dev dovecot: IMAP(br...@dev.example.com): rename(/mail/0/6/37/30/brian%dev.example.com/dovecot-uidlist.tmp, /mail/0/6/37/30/brian%dev.example.com/dovecot-uidlist) failed: No such file or directory Mar 25 10:22:48 freebsd-dev dovecot: IMAP(br...@dev.example.com): unlink(/mail/0/6/37/30/brian%dev.example.com/dovecot-uidlist.tmp) failed: No such file or directory Mar 25 10:22:36 wipe-dev dovecot: IMAP(br...@dev.example.com): ftruncate(/mail/0/6/37/30/brian%dev.example.com/dovecot-uidlist.lock) failed: Stale NFS file handle (Logs from a single test run are attached) Interestingly, these messages imply that dovecot is still using dotlocking in some circumstances, even though I've definitely set fcntl locking. $ grep ^lock /usr/local/etc/dovecot.conf lock_method = fcntl $ egrep '^mail_nfs|^mmap' /usr/local/etc/dovecot.conf mmap_disable = yes mail_nfs_storage = yes mail_nfs_index = yes All this suggests I should use some sort of 'sticky' load balancing in front so that all client conns from one IP hit the same frontend box. However, that contradicts the experience Adam McDougall has had with a similar setup: http://dovecot.org/list/dovecot/2010-March/047815.html It's possible that switching the Linux NFS server to a Netapp will help (which is what it will be deployed onto eventually anyway) Adam: did you do any tuning of FreeBSD client NFS settings? And have you tried using ImapTest, or just real IMAP users? I see there are a few tunables: $ grep nfs /etc/defaults/rc.conf netfs_types=nfs:NFS nfs4:NFS4 smbfs:SMB portalfs:PORTAL nwfs:NWFS # Net filesystems. nfs_client_enable=NO# This host is an NFS client (or NO). nfs_access_cache=60 # Client cache timeout in seconds nfs_server_enable=NO# This host is an NFS server (or NO). nfs_server_flags=-u -t -n 4 # Flags to nfsd (if enabled). nfs_reserved_port_only=NO # Provide NFS only on secure port
Re: [Dovecot] NFS issues
On Thu, Mar 25, 2010 at 05:25:53PM +0200, Timo Sirainen wrote:
Are you using ext4 on the Linux NFS server?
Yes:
/dev/sda1 on / type ext4 (rw,errors=remount-ro)
Nano/microsecond resolution timestamps fix the worst caching problems.
I can't see whether that's true through stat mtime though.
I've tried adding some extra debugging but it seems a variety of situations
are occurring:
Mar 25 15:49:36 freebsd-dev dovecot: IMAP(br...@dev.example.com): Our dotlock
file /mail/0/6/37/30/brian%dev.example.com/dovecot-uidlist.lock was deleted
(kept it 1 secs) file_dotlock_delete: unlink gave ENOENT
Mar 25 15:49:40 freebsd-dev dovecot: IMAP(br...@dev.example.com): Our dotlock
file /mail/0/6/37/30/brian%dev.example.com/dovecot-uidlist.lock was overridden
(kept it 0 secs) file_dotlock_delete: inode changed
Mar 25 15:49:54 freebsd-dev dovecot: IMAP(br...@dev.example.com): dotlock
/mail/0/6/37/30/brian%dev.example.com/dovecot-uidlist.lock was immediately
recreated under us
Mar 25 15:49:56 freebsd-dev dovecot: IMAP(br...@dev.example.com): dotlock
/mail/0/6/37/30/brian%dev.example.com/dovecot-uidlist.lock was immediately
recreated under us
Mar 25 15:50:18 freebsd-dev dovecot: IMAP(br...@dev.example.com): Our dotlock
file /mail/0/6/37/30/brian%dev.example.com/dovecot-uidlist.lock was overridden
(kept it 0 secs) file_dotlock_delete: inode changed
Mar 25 15:50:22 freebsd-dev dovecot: IMAP(br...@dev.example.com): dotlock
/mail/0/6/37/30/brian%dev.example.com/dovecot-uidlist.lock was immediately
recreated under us
Mar 25 15:50:27 freebsd-dev dovecot: IMAP(br...@dev.example.com): dotlock
/mail/0/6/37/30/brian%dev.example.com/dovecot-uidlist.lock was immediately
recreated under us
Out of interest, why is dot locking used instead of fcntl here?
Regards,
Brian.
--- src/lib/file-dotlock.c.orig 2010-03-25 15:40:12.0 +
+++ src/lib/file-dotlock.c 2010-03-25 15:48:17.0 +
@@ -671,7 +671,8 @@
return ret;
}
-static void dotlock_replaced_warning(struct dotlock *dotlock, bool deleted)
+static void dotlock_replaced_warning(struct dotlock *dotlock, bool deleted,
+ const char *debug)
{
const char *lock_path;
time_t now = time(NULL);
@@ -679,15 +680,17 @@
lock_path = file_dotlock_get_lock_path(dotlock);
if (dotlock-mtime == dotlock-lock_time) {
i_warning(Our dotlock file %s was %s
- (locked %d secs ago, touched %d secs ago),
+ (locked %d secs ago, touched %d secs ago) %s,
lock_path, deleted ? deleted : overridden,
(int)(now - dotlock-lock_time),
- (int)(now - dotlock-mtime));
+ (int)(now - dotlock-mtime),
+ debug ? debug : );
} else {
i_warning(Our dotlock file %s was %s
- (kept it %d secs), lock_path,
+ (kept it %d secs) %s, lock_path,
deleted ? deleted : overridden,
- (int)(now - dotlock-lock_time));
+ (int)(now - dotlock-lock_time),
+ debug ? debug : );
}
}
@@ -703,7 +706,9 @@
lock_path = file_dotlock_get_lock_path(dotlock);
if (nfs_safe_lstat(lock_path, st) 0) {
if (errno == ENOENT) {
- dotlock_replaced_warning(dotlock, TRUE);
+ dotlock_replaced_warning(dotlock, TRUE,
+file_dotlock_delete:
+nfs_safe_lstat gave ENOENT);
file_dotlock_free(dotlock);
return 0;
}
@@ -715,7 +720,8 @@
if (dotlock-ino != st.st_ino ||
!CMP_DEV_T(dotlock-dev, st.st_dev)) {
- dotlock_replaced_warning(dotlock, FALSE);
+ dotlock_replaced_warning(dotlock, FALSE,
+ file_dotlock_delete: inode changed);
errno = EEXIST;
file_dotlock_free(dotlock);
return 0;
@@ -731,7 +737,8 @@
if (unlink(lock_path) 0) {
if (errno == ENOENT) {
- dotlock_replaced_warning(dotlock, TRUE);
+ dotlock_replaced_warning(dotlock, TRUE,
+file_dotlock_delete: unlink gave ENOENT);
file_dotlock_free(dotlock);
return 0;
}
@@ -835,7 +842,8 @@
lock_path = file_dotlock_get_lock_path(dotlock);
if ((flags DOTLOCK_REPLACE_FLAG_VERIFY_OWNER) != 0
!file_dotlock_is_locked(dotlock)) {
- dotlock_replaced_warning(dotlock, FALSE);
+ dotlock_replaced_warning(dotlock, FALSE,
+ file_dotlock_replace: FLAG_VERIFY_OWNER failed);
errno = EEXIST;
file_dotlock_free(dotlock);
return 0;
@@ -844,7 +852,8 @@
if (rename(lock_path, dotlock-path) 0) {
i_error(rename(%s, %s) failed: %m, lock_path, dotlock-path);
if (errno == ENOENT)
- dotlock_replaced_warning(dotlock, TRUE);
+ dotlock_replaced_warning(dotlock, TRUE,
+file_dotlock_replace: rename gave ENOENT);
file_dotlock_free(dotlock);
return -1;
}
Re: [Dovecot] NFS issues
On Thu, 2010-03-25 at 16:04 +, Brian Candler wrote: Out of interest, why is dot locking used instead of fcntl here? My laziness. It's annoying to create different code for locking files via different ways in different places. (The files are created/written to in different ways, the locking can't simply be abstracted out into generic code. At least not too easily.) Of course the planned filesystem abstraction for v2.1/v3.0 should help with these problems in future. signature.asc Description: This is a digitally signed message part
[Dovecot] NFS issues [was: Dovecot-1.2 + Sieve + Managesieve on Debian]
On Tue, Mar 23, 2010 at 05:42:02AM -0400, Charles Marcus wrote: On 2010-03-22 9:31 PM, Stan Hoeppner wrote: Dovecot has built in locking support for NFS storage. But it has always been problematic, according to Timo. Have you got any references on this, apart from http://wiki.dovecot.org/NFS ? I'm looking at migrating a courier-imap installation (FreeBSD frontends, Netapp backends, Maildir++) to dovecot. I'd be grateful of any known pitfalls I should be looking out for. I have done some small-scale testing and it looks fine. tcpdumping the NFS traffic, I see that the FreeBSD frontend is sending access requests to check that its local cache is not stale. In tests with mailboxes containing 100 messages and a web IMAP frontend (atmail.org), Dovecot was generating about 1/4 of the total NFS traffic compared to courier-imap, because of how Dovecot creates a cache file containing the message headers. I see a link to http://www.freebsd.org/cgi/query-pr.cgi?pr=123755 in the NFS wiki page. Does this problem affect only mbox over NFS, or maildir too? I've not observed any problem with courier-imap, although courier-imap is much dumber about caching, and also at the moment the majority of the userbase are on POP3 anyway. Regards, Brian.
Re: [Dovecot] NFS issues [was: Dovecot-1.2 + Sieve + Managesieve on Debian]
On 23.3.2010, at 13.43, Brian Candler wrote: I have done some small-scale testing and it looks fine. Stress testing by running imaptest for same user's same mailbox in 2+ different servers (i.e. two NFS clients reading/writing same mailbox files) should show up quickly what kind of errors you could get. http://imapwiki.org/ImapTest
Re: [Dovecot] NFS issues [was: Dovecot-1.2 + Sieve + Managesieve on Debian]
On 03/23/10 07:43, Brian Candler wrote: On Tue, Mar 23, 2010 at 05:42:02AM -0400, Charles Marcus wrote: On 2010-03-22 9:31 PM, Stan Hoeppner wrote: Dovecot has built in locking support for NFS storage. But it has always been problematic, according to Timo. Have you got any references on this, apart from http://wiki.dovecot.org/NFS ? I'm looking at migrating a courier-imap installation (FreeBSD frontends, Netapp backends, Maildir++) to dovecot. I'd be grateful of any known pitfalls I should be looking out for. I have done some small-scale testing and it looks fine. tcpdumping the NFS traffic, I see that the FreeBSD frontend is sending access requests to check that its local cache is not stale. In tests with mailboxes containing 100 messages and a web IMAP frontend (atmail.org), Dovecot was generating about 1/4 of the total NFS traffic compared to courier-imap, because of how Dovecot creates a cache file containing the message headers. I see a link to http://www.freebsd.org/cgi/query-pr.cgi?pr=123755 in the NFS wiki page. Does this problem affect only mbox over NFS, or maildir too? I've not observed any problem with courier-imap, although courier-imap is much dumber about caching, and also at the moment the majority of the userbase are on POP3 anyway. Regards, Brian. I've used: mmap_disable: yes mail_nfs_storage: yes mail_nfs_index: yes on FreeBSD 6/7/8 and dovecot with multiple servers accessing the same mailboxes over NFS on a NetApp and it has worked fine since 1.0.16 or so, I think. I use 1.1 now (haven't finished testing 1.2 and haven't done much with 8.x). I have a load balancer in front and it sends IMAPS connections or HTTPS connections (making local IMAP calls) to randomized servers so even one person making 5 connections gets a speed benefit by using more than one server. If the NFS support had problems you could also use mail_location to keep indexes local.
