Re: [Dovecot] Is this really a user agent issue?
On Thu, 2010-12-23 at 15:08 -0700, Willie Gillespie wrote: The IMAP protocol has no way to push to the MUA that a folder (mailbox) has been freshly created. It has actually. NOTIFY extension makes it explicit, but it's possible to do that even in standard protocol. It's just that no server does it, and I don't know if any clients would actually do anything if a server did send it. I'm planning on having Dovecot send that some day once I've finally implemented mailbox list indexes.. Anyway, if a mailbox foo is created, a server can simply send to clients: * LIST (\HasNoChildren) / foo signature.asc Description: This is a digitally signed message part
Re: [Dovecot] Is this really a user agent issue?
On Thu, Dec 23, 2010 at 17:08, Willie Gillespie wgillespie+dove...@es2eng.com wrote: Phil Howard wrote: I think this issue has been entirely misunderstood. Have I explained it wrong? I think there's been a bit of confusion here. Everyone is saying similar things in slightly different ways. The IMAP protocol has no way to push to the MUA that a folder (mailbox) has been freshly created. This information must be pulled by the client, ie: LISTing all folders. Given that it appeared to be a request/response class protocol, I was expecting that it had no such push ability. So, say we have an MUA connected and -something- (whether it be deliver or another MUA connected elsewhere) creates a new folder. With IMAP the original MUA has no way of knowing that this just happened. When it tries to create the same folder, the CREATE fails, because the folder already exists. Right. How the MUA handles this situation is up to the MUA. I see a few possibilities: 1) it could ignore the situation and just show an error message to the user* 2) it could do a LIST and get an updated list of folders** 3) it could add the folder to its display*** * Sounds like what your MUA is doing. Yes. ** This could be fine and dandy, but many MUAs use the subscription list (LSUB) instead of showing all the folders (LIST). So just because the MUA now knows the folder exists doesn't mean it will show it to you unless you SUBSCRIBE to it. However, if I am not doing subscriptions, shouldn't it show me ALL folders (per what Charles Marcus said in his message just before yours)? Why would this folder be handled differently if it is showing me all the other folders? *** Whether this means that the MUA auto-SUBSCRIBEs you to the mailbox or not depends on what mode the MUA is running in. It seems like this is what you want your MUA to do instead of #1. No, that is not what I want it to do. What I want it to do is #2 ... and show me ALL the folders, with the new one included from the most recent LIST. It should do LIST as a result of there being an error from CREATE ... to determine if the error was because the folder had been created by other than the MUA. If I were a programmer, #1 would definitely be the easiest to do. Then I wouldn't have to care WHY the CREATE failed, I just show an error message no matter what. But #2 is not really harder. It's another step. I don't think of such logic has harder. To me, hardness of programming is the difficulty level of figuring out what algorithm to use ... e.g. what works and is expected to work. I would do #2. If as a programmer I was trying to make it easier, I'd just not write any of it at all. When I do programming, though, I consider that the effort to meet reasonable human expectations is part of the job/project. If the developer believes humans expect to not be able to get to a folder because it had previously been created by something else, it should at least be informative ... Sorry, you cannot access folders that were created by other than you, without restarting the client. It's just so much simpler, even with the need to do a whole LIST request, to give the human the realistic expectation of seeing the folder show up after a folder creation dialog, regardless if something else created it first. So in answer to the question in the subject, Is this really a user agent issue? Yes. The server is doing nothing wrong according to protocol. I really didn't think it was. But I was wondering if there was some possibility the IMAP protocol had a limitation that completely prevented this (e.g. it wouldn't be in the LIST response, either, or LIST is only allowed once when connecting, or whatever). It appears IMAP is a minimal but reasonable protocol, and provides sufficient means for MUA logic to be reasonable, and Evolution fell short of that. I wanted to be sure that assumption was correct. -- sHiFt HaPpEnS!
Re: [Dovecot] Is this really a user agent issue?
I think this issue has been entirely misunderstood. Have I explained it wrong? 2010/12/22 Karsten Bräckelmann guent...@rudersport.de: On Wed, 2010-12-22 at 09:34 -0500, Phil Howard wrote: 2010/12/21 Karsten Bräckelmann guent...@rudersport.de: Creating the new mail folder is entirely on the IMAP server side. The MUA (Evolution in your stated case) is irrelevant. If the creation of the new folder fails, it is a server side problem. However, once a new folder has been created (server side, mind you), Evolution won't know about that folder until it is restarted. Probably the same with other MUAs, too. In the worst case, restarting Evo twice should show the new folder. (That is assuming you are not limiting your MUA to subscribed folders only, or, as IIRC is the default, deliver auto-subscribes the user to the just created folder.) Then I would call this a user agent issue. You snipped the part of your OP where you stated that dovecot deliver fails creating a mail folder. This is NOT an MUA issue. Then whose issue it is? There is clearly an issue because an existing folder cannot be accessed by this MUA. If IMAP provides a way for the user agent to discover the folder already exists, then the user agent should do this, at least if an error is encountered when trying to create it. Evolution clearly does not (or doesn't act on knowing if it does). Other user agents, I don't know. But it is simple logic: There is a way for the MUA to discover folders existing, LIST. Existing on the IMAP server, mind you, again. So this is not an MUA limitation. But it would be an MUA issue if the MUA fails to do LIST after an error from CREATE or whatever command it is. Also, MUAs don't care about the full list of folders, if they are in subscription mode. In that case, they ask for subscribed folders, LSUB, not all folders. It was not in subscription mode. Which method do you use? (That's a client option.) Whatever it starts out as. I don't do subscriptions because so far it hasn't needed to do that. I don't want a subset of folders. I want all folders. How would subscription help there? if (create folder $name succeeds) OR (folder $name can be accessed) then set up local reference to folder $name This is unrelated. *sigh* In your case, the creation is server side. And yes, unless you use subscription only, Evo does exactly that on the next start -- ask for all folders on the server. The point is, if it tries to create a folder, and there is a failure, and because a developer should know and understand that an existing folder is a cause for a failure of a create operation, it should try to distinguish this. A user expects the folder to show up in the list when created when there is no error. If the folder already exists, then there is no valid reason to not put that folder in the list now, just as a successful create would do. Since the protocol apparently provides no means to reflect this situation accurately, the MUA needs to have some means to get access to the existing folder. Restarting the MUA is not an acceptable means. else report error about creating folder $name endif Impossible. The error creating the folder occurred on the server, during delivery, which does not even need to be doveot deliver but any number of other MDAs. No. The delivery successfully created the folder. The error happened later when the MUA tried to create the same folder, due to the MUA not being aware the folder had been created during delivery. But in the case of Evolution, because of that failure in the create operation, it did not allow any access to that folder. The existing folder it tried to create did not get added to the list. It could have been had the logic I described been used. Or it could have been had IMAP provided for a better explain error (more often referred to as an exceptional condition in the classic programming methods I originally learned, rather than a pure unexplained error). Still, the MUA could get around that with the suggested logic. Either way, no client needs to be running while mail is being delivered, and often is not. The server just does not even tell the client about a failure, even *if* it potentially could know about that. This was purely a server-side operation and failure, and the client, any client, is not going to be informed about it. Only a successfully created folder will be known to the MUA during the LIST command. If the folder was created by delivery of mail after the MUA was started, and then how do you think an MUA should become aware of that folder ... other than by restarting the MUA which is not an acceptable requirement. That, of course, depends on IMAP having a means to detect if the folder exists (even though the folder was not given in a previous list of existing folders). Trying to open it should be a way to do that test. Yes, the IMAP server does detect new
Re: [Dovecot] Is this really a user agent issue?
Out of curiosity, are you using the autosubscribe feature of Dovecot LDA to automatically subscribe people who whatever new folder the Dovecot LDA automatically creates? It looks like this is the -s flag to LDA binary in 1.1.3+ or lda_mailbox_autosubscribe in 2.0+ (not sure exactly what point release...) I don't have a lot of experience with Evolution but I know Thunderbird can be difficult when it comes to new folders showing up on the server that it didn't create. The best you can do is make sure the client gets automatically subscribed to that new folder and hope that the client checks the list of subscribed folders (the LSUB IMAP command) eventually. I would hope that clicking check/get new mail would be enough to trigger that in most clients. On Tue, 21 Dec 2010 14:48:30 -0500, Phil Howard ttip...@gmail.com wrote: I suspect this is a user agent issue, given that Evolution is flaky in so many areas. When I deliver mail to a subfolder/subbox (e.g. the -m option in the deliver command), and Evolution doesn't know of it, yet, creating it fails, and Evolution still can't get to it. Doing things the other way around (create it in Evolution first, then deliver to it) works fine. Seems to be silliness to me. Just wondered if any Dovecot aspects might be involved ... such as the fact that the case is different (e.g. INBOX in the filesystem, but Inbox in Evolution).
Re: [Dovecot] Is this really a user agent issue?
On Thu, Dec 23, 2010 at 10:27, David Warden war...@geneseo.edu wrote: Out of curiosity, are you using the autosubscribe feature of Dovecot LDA to automatically subscribe people who whatever new folder the Dovecot LDA automatically creates? It looks like this is the -s flag to LDA binary in 1.1.3+ or lda_mailbox_autosubscribe in 2.0+ (not sure exactly what point release...) I didn't purposely use it. I see nothing in /etc/dovecot/dovecot.conf that suggests that. I see no -s flag on any running binary. There is no -s flag configured for deliver in Postfix's master.cf. So I conclude that answer to be no. I don't have a lot of experience with Evolution but I know Thunderbird can be difficult when it comes to new folders showing up on the server that it didn't create. The best you can do is make sure the client gets automatically subscribed to that new folder and hope that the client checks the list of subscribed folders (the LSUB IMAP command) eventually. I would hope that clicking check/get new mail would be enough to trigger that in most clients. I don't know the semantics of this, and would not want to suddenly change things on users. Maybe they are doing subscribes on their own that might highlight what they subscribe to and ignore all else ... and that doing autosubscribe might interfere with that. Evolution and Thunderbird seem to be fine with no subscriptions going on. On startup they seem to find all the folders. The issue I'm encountering (tried only on Evolution since that is what I use right now) is that when I want to access a folder that was created during delivery, I cannot. There is no folder shown and no option to refresh it (unless doing a subscribe to an as-yet unseen folder would do it, after which I could then unsubscribe) short of quitting Evolution and restarting. Doing Send/Receive doesn't refresh (I didn't really expect it to, but I did try it just in case). The most obvious option is doing a right button click on the parent folder and selecting refresh. That didn't add the folder, either, so I conclude it only refreshes the count of emails. So I tried to create it within Evolution. I figured that would at least set up the folder name in the tree on the left panel. But, since IMAP reported a failure to create (because it existed at that time, due to the earlier create during delivery), Evolution just gave up and did not add the folder. I believe what it should do when creating a folder fails is to check to see if that folder already exists, and if so, either add the folder anyway, or ask the user if the already created folder should be added. When programming in C, when I want to create a file in a directory, and want to create the directory if it does not yet exist, what I code is a mkdir() call with error handling that ignores EEXIST. The intended directory name might be something other than a directory. But I know that open() naming a file in that directory to be created, will fail if it wasn't a directory. So I do not need to explicitly stat() the directory to see if it is a directory or not. If open() fails, then I sort out the error to the level that program calls for. Such a program would create the file in that directory whether the directory exists or not. Extending this concept to a program using a protocol like IMAP ... I don't know if that's beyond the ability of the application developer(s) to grasp, or not. But it is the case that I find very little goofy logic in systems and server daemons, and find a lot of that in GUI applications. I just didn't want to jump to even an obvious conclusion in this case, knowing that I did not know details about IMAP (if I had known IMAP, I probably would have come to some conclusion, quickly). -- sHiFt HaPpEnS!
Re: [Dovecot] Is this really a user agent issue?
On 2010-12-23 10:27 AM, David Warden wrote: I know Thunderbird can be difficult when it comes to new folders showing up on the server that it didn't create. Main reason I always uncheck the 'show only subscribed folders' option in the Advanced settings... -- Best regards, Charles
Re: [Dovecot] Is this really a user agent issue?
On Thu, Dec 23, 2010 at 12:28, Charles Marcus cmar...@media-brokers.com wrote: On 2010-12-23 10:27 AM, David Warden wrote: I know Thunderbird can be difficult when it comes to new folders showing up on the server that it didn't create. Main reason I always uncheck the 'show only subscribed folders' option in the Advanced settings... If folders were automatically subscribed when created, and while the user agent was running and connected to imapd, would there be a notification sent? Or does the user agent in such a case always ask for all subscribed folders and see the new one at that time. It would not help if the new folder did not show up. -- sHiFt HaPpEnS!
Re: [Dovecot] Is this really a user agent issue?
On 2010-12-23 2:33 PM, Phil Howard wrote: On Thu, Dec 23, 2010 at 12:28, Charles Marcus wrote: On 2010-12-23 10:27 AM, David Warden wrote: I know Thunderbird can be difficult when it comes to new folders showing up on the server that it didn't create. Main reason I always uncheck the 'show only subscribed folders' option in the Advanced settings... If folders were automatically subscribed when created, and while the user agent was running and connected to imapd, would there be a notification sent? Or does the user agent in such a case always ask for all subscribed folders and see the new one at that time. It would not help if the new folder did not show up. By UNchecking the option, you are basically saying I don't care what folders are subscribed or not, show me ALL of them. -- Best regards, Charles
Re: [Dovecot] Is this really a user agent issue?
Phil Howard wrote: I think this issue has been entirely misunderstood. Have I explained it wrong? I think there's been a bit of confusion here. Everyone is saying similar things in slightly different ways. The IMAP protocol has no way to push to the MUA that a folder (mailbox) has been freshly created. This information must be pulled by the client, ie: LISTing all folders. So, say we have an MUA connected and -something- (whether it be deliver or another MUA connected elsewhere) creates a new folder. With IMAP the original MUA has no way of knowing that this just happened. When it tries to create the same folder, the CREATE fails, because the folder already exists. How the MUA handles this situation is up to the MUA. I see a few possibilities: 1) it could ignore the situation and just show an error message to the user* 2) it could do a LIST and get an updated list of folders** 3) it could add the folder to its display*** * Sounds like what your MUA is doing. ** This could be fine and dandy, but many MUAs use the subscription list (LSUB) instead of showing all the folders (LIST). So just because the MUA now knows the folder exists doesn't mean it will show it to you unless you SUBSCRIBE to it. *** Whether this means that the MUA auto-SUBSCRIBEs you to the mailbox or not depends on what mode the MUA is running in. It seems like this is what you want your MUA to do instead of #1. If I were a programmer, #1 would definitely be the easiest to do. Then I wouldn't have to care WHY the CREATE failed, I just show an error message no matter what. So in answer to the question in the subject, Is this really a user agent issue? Yes. The server is doing nothing wrong according to protocol.
Re: [Dovecot] Is this really a user agent issue?
2010/12/21 Karsten Bräckelmann guent...@rudersport.de: Creating the new mail folder is entirely on the IMAP server side. The MUA (Evolution in your stated case) is irrelevant. If the creation of the new folder fails, it is a server side problem. However, once a new folder has been created (server side, mind you), Evolution won't know about that folder until it is restarted. Probably the same with other MUAs, too. In the worst case, restarting Evo twice should show the new folder. (That is assuming you are not limiting your MUA to subscribed folders only, or, as IIRC is the default, deliver auto-subscribes the user to the just created folder.) Then I would call this a user agent issue. If IMAP provides a way for the user agent to discover the folder already exists, then the user agent should do this, at least if an error is encountered when trying to create it. Evolution clearly does not (or doesn't act on knowing if it does). Other user agents, I don't know. But it is simple logic: if (create folder $name succeeds) OR (folder $name can be accessed) then set up local reference to folder $name else report error about creating folder $name endif That, of course, depends on IMAP having a means to detect if the folder exists (even though the folder was not given in a previous list of existing folders). Trying to open it should be a way to do that test. But if the IMAPD process itself can't or won't even try to access that folder because it wasn't in the list when it started, then I see that as a server side issue, or if the protocol doesn't provide any such means to ask for a folder not previously seen, then I see that as a protocol design issue. -- sHiFt HaPpEnS!
Re: [Dovecot] Is this really a user agent issue?
On Wed, 2010-12-22 at 09:34 -0500, Phil Howard wrote:
2010/12/21 Karsten Bräckelmann guent...@rudersport.de:
Creating the new mail folder is entirely on the IMAP server side. The
MUA (Evolution in your stated case) is irrelevant. If the creation of
the new folder fails, it is a server side problem.
However, once a new folder has been created (server side, mind you),
Evolution won't know about that folder until it is restarted. Probably
the same with other MUAs, too. In the worst case, restarting Evo twice
should show the new folder.
(That is assuming you are not limiting your MUA to subscribed folders
only, or, as IIRC is the default, deliver auto-subscribes the user to
the just created folder.)
Then I would call this a user agent issue.
You snipped the part of your OP where you stated that dovecot deliver
fails creating a mail folder. This is NOT an MUA issue.
If IMAP provides a way for
the user agent to discover the folder already exists, then the user
agent should do this, at least if an error is encountered when trying
to create it. Evolution clearly does not (or doesn't act on knowing
if it does). Other user agents, I don't know. But it is simple
logic:
There is a way for the MUA to discover folders existing, LIST. Existing
on the IMAP server, mind you, again. So this is not an MUA limitation.
Also, MUAs don't care about the full list of folders, if they are in
subscription mode. In that case, they ask for subscribed folders, LSUB,
not all folders.
Which method do you use? (That's a client option.)
if (create folder $name succeeds) OR (folder $name can be accessed) then
set up local reference to folder $name
This is unrelated. *sigh* In your case, the creation is server side.
And yes, unless you use subscription only, Evo does exactly that on the
next start -- ask for all folders on the server.
else
report error about creating folder $name
endif
Impossible. The error creating the folder occurred on the server, during
delivery, which does not even need to be doveot deliver but any number
of other MDAs.
Either way, no client needs to be running while mail is being delivered,
and often is not. The server just does not even tell the client about a
failure, even *if* it potentially could know about that. This was purely
a server-side operation and failure, and the client, any client, is not
going to be informed about it.
Only a successfully created folder will be known to the MUA during the
LIST command.
That, of course, depends on IMAP having a means to detect if the
folder exists (even though the folder was not given in a previous list
of existing folders). Trying to open it should be a way to do that
test.
Yes, the IMAP server does detect new folders.
Trying to open it sounds like a client action -- in which case it is
just wrong, because the client cannot even try opening something it
never knew before existed. Other than brute-force trying all possible
char combinations...
But if the IMAPD process itself can't or won't even try to access that
folder because it wasn't in the list when it started, then I see that
as a server side issue, or if the protocol doesn't provide any such
means to ask for a folder not previously seen, then I see that as a
protocol design issue.
Wow, you really do know what you're talking about, don't you? *sigh*
In a nutshell: Failure to create a folder at delivery time is NOT an MUA
issue. And whether or not the user will see new folders created
successfully depends on (not) limiting to subscribed folders, and the
subscription list stored on the server.
Basically, exactly what I wrote in my first reply. Mind reading that
carefully again?
--
char *t=\10pse\0r\0dtu...@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128 (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
[Dovecot] Is this really a user agent issue?
I suspect this is a user agent issue, given that Evolution is flaky in so many areas. When I deliver mail to a subfolder/subbox (e.g. the -m option in the deliver command), and Evolution doesn't know of it, yet, creating it fails, and Evolution still can't get to it. Doing things the other way around (create it in Evolution first, then deliver to it) works fine. Seems to be silliness to me. Just wondered if any Dovecot aspects might be involved ... such as the fact that the case is different (e.g. INBOX in the filesystem, but Inbox in Evolution). -- sHiFt HaPpEnS!
Re: [Dovecot] Is this really a user agent issue?
On Tue, 2010-12-21 at 14:48 -0500, Phil Howard wrote:
I suspect this is a user agent issue, given that Evolution is flaky in
so many areas. When I deliver mail to a subfolder/subbox (e.g. the -m
option in the deliver command), and Evolution doesn't know of it, yet,
creating it fails, and Evolution still can't get to it. Doing things
Creating the new mail folder is entirely on the IMAP server side. The
MUA (Evolution in your stated case) is irrelevant. If the creation of
the new folder fails, it is a server side problem.
However, once a new folder has been created (server side, mind you),
Evolution won't know about that folder until it is restarted. Probably
the same with other MUAs, too. In the worst case, restarting Evo twice
should show the new folder.
(That is assuming you are not limiting your MUA to subscribed folders
only, or, as IIRC is the default, deliver auto-subscribes the user to
the just created folder.)
the other way around (create it in Evolution first, then deliver to
it) works fine. Seems to be silliness to me. Just wondered if any
Dovecot aspects might be involved ... such as the fact that the case
is different (e.g. INBOX in the filesystem, but Inbox in
Evolution).
--
char *t=\10pse\0r\0dtu...@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128 (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
