Re: [Dovecot] graceful failure when some folders are not available...
On Sat, Oct 20, 2007 at 10:10:00PM +0300, Timo Sirainen wrote: > On Mon, 2007-10-08 at 16:39 -0500, Steven F Siirila wrote: > > And in fact your tests (below) reproduced this. The problem with this is > > that if even one file or directory within the user's IMAP folder space is > > currently unavailable (due to an NFS server being down), the user cannot > > log in at all to access any of their other folders. In out scenario, we > > would prefer that the user simply not see the folders (treat the error the > > same as "file not found"). BTW, the errno seen is ETIMEDOUT (we are soft > > mounting the NFS filesystem in question). Any thoughts on how we can > > accomplish this? We don't normally expect this NFS filesystem to become > > unavailable, but when it is, we don't want it to prevent all users from > > being able to log in, since this NFS filesystem only holds folders of an > > archival nature. > > Does the attached patch help? > > It should work pretty nicely if index files are still available. If they > aren't, it shows the mailbox as being empty. In conjunction with this inquiry, we attempted an alternate method. We created a new namespace for the hierarchy which was NFS-mounted. In the IMAP startup script we checked to see if the NFS mount was accessible. If it was available, we'd enable this namespace. Otherwise, we would not. In this way we could avoid the behavior exhibited at Dovecot startup where login would fail if the NFS mount was unavailable. In the case of existing sessions, the user will still see a 1 minute delay before getting an error back, but we can live with that. So, in short, we did not test the provided patch (sorry). We realized that using this patch alone would have still left us with problems in cases where many files were in the NFS-mounted hierarchy, as it would have had to stat() each (and timeout on each). Thanks for the patch nonetheless! -- Steven F. Siirila Office: Univ Park Plaza, Room 750 Internet Services E-mail: [EMAIL PROTECTED] Office of Information TechnologyVoice: (612) 626-0244 University of Minnesota Fax: (612) 626-7593
Re: [Dovecot] graceful failure when some folders are not available...
On Mon, 2007-10-08 at 16:39 -0500, Steven F Siirila wrote: > And in fact your tests (below) reproduced this. The problem with this is > that if even one file or directory within the user's IMAP folder space is > currently unavailable (due to an NFS server being down), the user cannot > log in at all to access any of their other folders. In out scenario, we > would prefer that the user simply not see the folders (treat the error the > same as "file not found"). BTW, the errno seen is ETIMEDOUT (we are soft > mounting the NFS filesystem in question). Any thoughts on how we can > accomplish this? We don't normally expect this NFS filesystem to become > unavailable, but when it is, we don't want it to prevent all users from > being able to log in, since this NFS filesystem only holds folders of an > archival nature. Does the attached patch help? It should work pretty nicely if index files are still available. If they aren't, it shows the mailbox as being empty. diff -r ed09701ff3d0 src/lib-storage/index/mbox/mbox-sync.c --- a/src/lib-storage/index/mbox/mbox-sync.c Sat Oct 20 22:00:57 2007 +0300 +++ b/src/lib-storage/index/mbox/mbox-sync.c Sat Oct 20 22:08:02 2007 +0300 @@ -1496,6 +1496,12 @@ int mbox_sync_has_changed(struct mbox_ma } } else { if (stat(mbox->path, &statbuf) < 0) { + if (errno == ETIMEDOUT && + strcmp(mbox->ibox.box.name, "INBOX") != 0) { +/* NFS server down? Ignore for all mailboxes + except INBOX. */ +return 0; + } mbox_set_syscall_error(mbox, "stat()"); return -1; } signature.asc Description: This is a digitally signed message part
Re: [Dovecot] graceful failure when some folders are not available...
Did I miss a reply on this? We're considering modifying Dovecot, but would like opinions before going down the wrong path... On Mon, Oct 08, 2007 at 04:39:25PM -0500, Steven F Siirila wrote: > On Sat, Oct 06, 2007 at 04:28:20AM +0300, Timo Sirainen wrote: > > On Wed, 2007-10-03 at 20:03 -0500, [EMAIL PROTECTED] wrote: > > > Hi folks. Quick question in the hopes that someone knows the answer, > > > before > > > I dig in the code some more. > > > > > > In testing a new setup with some long-term archival mbox-format mailboxes > > > stored on an NFS mount, we've found the following: if the mount is > > > unavailable for any reason, the user cannot log into their email at all. > > > Dovecot says: "stat() failed with mbox foo" and dies. This is coming from > > Perhaps "dies" was too strong. In fact, Dovecot does not die, but the > client perceives such as it is told this upon trying to log in: > > "The current command did not succeed. The mail server responded: >Internal error occurred. Refer to server log for more information." > > And in fact your tests (below) reproduced this. The problem with this is > that if even one file or directory within the user's IMAP folder space is > currently unavailable (due to an NFS server being down), the user cannot > log in at all to access any of their other folders. In out scenario, we > would prefer that the user simply not see the folders (treat the error the > same as "file not found"). BTW, the errno seen is ETIMEDOUT (we are soft > mounting the NFS filesystem in question). Any thoughts on how we can > accomplish this? We don't normally expect this NFS filesystem to become > unavailable, but when it is, we don't want it to prevent all users from > being able to log in, since this NFS filesystem only holds folders of an > archival nature. > > > > the mbox sync checks. (It's possible the same happens with a maildir > > > folder--I'm just specifying mbox because that's what we've tested with so > > > far). > > > > It shouldn't die. Maybe your client kills the connection? > > > > I tested this by making the stat() call always fail with EIO: > > > > x select inbox > > x NO Internal error occurred. Refer to server log for more information. > > [2007-10-06 04:24:48] > > x status foo (messages) > > x NO Internal error occurred. Refer to server log for more information. > > [2007-10-06 04:24:58] > > > > Or even if the mailbox is successfully opened and after that: > > > > x noop > > * NO Internal error occurred. Refer to server log for more information. > > [2007-10-06 04:27:31] > > x OK NOOP completed. > > > > > > -- > > Steven F. Siirila Office: Univ Park Plaza, Room 750 > Internet Services E-mail: [EMAIL PROTECTED] > Office of Information Technology Voice: (612) 626-0244 > University of Minnesota Fax: (612) 626-7593 -- Steven F. Siirila Office: Univ Park Plaza, Room 750 Internet Services E-mail: [EMAIL PROTECTED] Office of Information TechnologyVoice: (612) 626-0244 University of Minnesota Fax: (612) 626-7593
Re: [Dovecot] graceful failure when some folders are not available...
On Sat, Oct 06, 2007 at 04:28:20AM +0300, Timo Sirainen wrote: > On Wed, 2007-10-03 at 20:03 -0500, [EMAIL PROTECTED] wrote: > > Hi folks. Quick question in the hopes that someone knows the answer, before > > I dig in the code some more. > > > > In testing a new setup with some long-term archival mbox-format mailboxes > > stored on an NFS mount, we've found the following: if the mount is > > unavailable for any reason, the user cannot log into their email at all. > > Dovecot says: "stat() failed with mbox foo" and dies. This is coming from Perhaps "dies" was too strong. In fact, Dovecot does not die, but the client perceives such as it is told this upon trying to log in: "The current command did not succeed. The mail server responded: Internal error occurred. Refer to server log for more information." And in fact your tests (below) reproduced this. The problem with this is that if even one file or directory within the user's IMAP folder space is currently unavailable (due to an NFS server being down), the user cannot log in at all to access any of their other folders. In out scenario, we would prefer that the user simply not see the folders (treat the error the same as "file not found"). BTW, the errno seen is ETIMEDOUT (we are soft mounting the NFS filesystem in question). Any thoughts on how we can accomplish this? We don't normally expect this NFS filesystem to become unavailable, but when it is, we don't want it to prevent all users from being able to log in, since this NFS filesystem only holds folders of an archival nature. > > the mbox sync checks. (It's possible the same happens with a maildir > > folder--I'm just specifying mbox because that's what we've tested with so > > far). > > It shouldn't die. Maybe your client kills the connection? > > I tested this by making the stat() call always fail with EIO: > > x select inbox > x NO Internal error occurred. Refer to server log for more information. > [2007-10-06 04:24:48] > x status foo (messages) > x NO Internal error occurred. Refer to server log for more information. > [2007-10-06 04:24:58] > > Or even if the mailbox is successfully opened and after that: > > x noop > * NO Internal error occurred. Refer to server log for more information. > [2007-10-06 04:27:31] > x OK NOOP completed. > -- Steven F. Siirila Office: Univ Park Plaza, Room 750 Internet Services E-mail: [EMAIL PROTECTED] Office of Information TechnologyVoice: (612) 626-0244 University of Minnesota Fax: (612) 626-7593
Re: [Dovecot] graceful failure when some folders are not available...
On Wed, 2007-10-03 at 20:03 -0500, [EMAIL PROTECTED] wrote: > Hi folks. Quick question in the hopes that someone knows the answer, before > I dig in the code some more. > > In testing a new setup with some long-term archival mbox-format mailboxes > stored on an NFS mount, we've found the following: if the mount is > unavailable for any reason, the user cannot log into their email at all. > Dovecot says: "stat() failed with mbox foo" and dies. This is coming from > the mbox sync checks. (It's possible the same happens with a maildir > folder--I'm just specifying mbox because that's what we've tested with so > far). It shouldn't die. Maybe your client kills the connection? I tested this by making the stat() call always fail with EIO: x select inbox x NO Internal error occurred. Refer to server log for more information. [2007-10-06 04:24:48] x status foo (messages) x NO Internal error occurred. Refer to server log for more information. [2007-10-06 04:24:58] Or even if the mailbox is successfully opened and after that: x noop * NO Internal error occurred. Refer to server log for more information. [2007-10-06 04:27:31] x OK NOOP completed. signature.asc Description: This is a digitally signed message part
[Dovecot] graceful failure when some folders are not available...
Hi folks. Quick question in the hopes that someone knows the answer, before I dig in the code some more. In testing a new setup with some long-term archival mbox-format mailboxes stored on an NFS mount, we've found the following: if the mount is unavailable for any reason, the user cannot log into their email at all. Dovecot says: "stat() failed with mbox foo" and dies. This is coming from the mbox sync checks. (It's possible the same happens with a maildir folder--I'm just specifying mbox because that's what we've tested with so far). Is there a way to reconfigure this behavior? I could maybe see a fatal abort if the inbox is unavailable, but for other folders it seems rather... presumptuous. I have to think there's already a way to handle this more gracefully in the config and I'm just not seeing it. Also, does anyone know offhand if this behavior is the same for folders that aren't in the default/inbox namespace? That would seem *really* wrong. Any thoughts? Thanks much, -Brian