Comments on your message:
[1] There is no such operation as "move". What you think is a "move" is
actually implemented by two separate operations:
copy from source to destination
delete from source
and optionally a third:
expunge (permanently remove deleted messages from) source
[2] Your report of "sometimes the move is partly honors in that the item
appears in the target but is not deleted from the source" is explained as
the first operation completing successfully, but not the second operation.
[3] Your report of "sometimes the move is not honored at all" is explained
as the first operation never taking place.
[4] UW IMAP uses /tmp, not /usr/tmp.
[5] It sounds like you are using traditional UNIX mailbox format, a.k.a.
"mbox format"; a flat-file containing all the messages, one after another,
with each message preceded with a line starting with "From ". If this is
the case: traditional UNIX mailbox format was designed in a time when a
"very large mailbox" was between 100 KB and 800 KB in size. It does not
scale well to sizes of 100 MB to 800 MB.
In particular, doing things with mailboxes in the hundreds of MB in that
format takes a while. The authors of Outlook and Thunderbird are victims
of a computer science course mindset which, starting in the 1980s, taught
their pupils that all protocols are (or should be) stateless. Thus, they
believe that IMAP is like HTTP; that when a server fails to respond
immediately, that means that the correct remedial action is to disconnect
and try again, or just disconnect and assume that everything happened
anyway.
These developers simply can not grasp the concept of a stateful protocol
like IMAP (HTTP is stateless) in which certain operations (such as
manipulations of an 800MB flat file) might take a while; and that being a
stateful protocol IMAP is guaranteed to respond if you wait long enough.
Unfortunately, years of repeated attempts to educate these developers
proved to be utterly futile. When you talk about "state", you get vacant
stares.
Thus, there is no hope of a working version of crapware such as Outlook or
Thunderbird. The only thing that you can do is see what you can do on the
server to not unduly stress the limitations of the crapware.
One way to do accomplish this is to switch mailbox formats to mix format.
I can't imagine how anyone can tolerate traditional UNIX mailbox format
with a 100 MB mailbox, much less larger. mix format was designed to
accomodate such large mailboxes; and does in a matter of a second or two
what can take minutes with traditional UNIX mailbox format.
[6] What you describe is typical of scaling problems. It seems to "work"
until you reach a critical point. Very likely, the mailboxes got bit
enough that operations now tend to take long enough for Thunderbird to
decide to give up.
[7] Both Thunderbird and Outlook have a way to record protocol
negotiations. The c-client library also has a mechanism to record
protocol negotiations, at the client (not server) level.
You want to do client-based telemetry, not server-based, due to the size
of the protocol logs. IMAP protocol logs are HUGE, and on anything beyond
a small test system you will quickly be buried under the weight of the
logs. On a small test system, you can easily set up a script for server
logs using "tee", at the cost of giving up SSL capability.
You quickly find out that the server logs are much less useful than you
thought. I went through this exercise on my new IMAP server.
On Fri, 30 Apr 2010, support wrote:
We have users whose folders are between 100 MB and 800 MB in size.
Most of those users are using Outlook but some are using Thunderbird.
Lately (and seemingly suddenly) the users are encountering trouble in that
they will move one or more items from the inbox to another folder and get
inconsistent results:
* sometimes the move is honored, with the item appearing in the target
folder and disappearing from the source folder.
* sometimes the move is partly honored in that the item appears in the
target folder, but is NOT deleted from the source folder.
* and sometimes the move is not honored at all, despite the client email
program thinking for a moment that it was (before seeing in moments
after that it wasn't.)
The server is FreeBSD 5.4 using imap2006e, I think. I'll upgrade to
imap2007, whatever's current in the FreeBSD ports tree, to see if it
helps.
(1) I insured that /var/mail and /usr/tmp were permission 1777 as
suggested, but just now so I don't know if that will prove to help.
Things used to work just fine without these changes; the only things
that may have changed are (a) the users' mail folders have gotten
larger, (b) the versions of Outlook they are using are new. Thunderbird
(the latest in general) also suffers the failed transfers, though, and
only "suddenly" now for no apparent reason.
(2) I see no reference to debugging on the website. I thought I could
perhaps turn on a flag for imap in inetd to be able to track requests to
try to see what the server was being told to do, but I see no mention of
any such facility for either imap or c-client.
-- Mark --
http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.
_______________________________________________
Imap-uw mailing list
Imap-uw@u.washington.edu
http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
