On Fri, 20 Feb 2009, Shawn Walker wrote:
Sheesh, how slow is the link?
Not everybody has a fast link.
Yes, but there is slow and there is slow; just as there is fast and there
is fast.
Algorithms which may be suitable for 300 bps are dreadful for 2G wireless
(EDGE or 1x). Algorithms which may be suitable for 2G wireless are less
good for 3G (UMTS or EV-DO).
I have found that people spend time worrying about bandwidth issues that
are ultimately unimportant, and end up blundering into the hell of
excessive RTTs.
When you talk about fetching 25 UIDs at a time, that is a clue to me that
you are about to blunder into RTT hell. That's a suitable number for a
bandwidth of 1200 bps. Now, you may be in a time machine that took you
back 20 years; but otherwise you are almost certainly have much faster
bandwidth.
RTTs are deadly. Just watch how long it takes Alpine to gobble a large
attachment -- it's due to the RTTs introduced by breaking up a big
download into smaller chunks to allow the user to stop the download.
Mind you; the Alpine guys had a clue of what a good chunk size would be;
yet if you have any kind of broadband it is almost always better to set
"Prevent Partial Fetching".
I'm not sure why I didn't think of using message numbers instead of UIDs.
Duh! I'll be using that.
A lot of people fail to understand the important properties of message
sequence numbers, including those who advocate a "simplified" version of
IMAP that abolishes sequence numbers (while meanwhile adding all sorts of
new fluff...witness all the silly new IMAP extensions defined in recent
RFCs).
It's the perception that to the end user that the product is downloading the
messages instead of having the user think the application isn't doing
anything for several minutes. I'll be playing around what the number would
be for getting the uids in a batch.
You need to do SUBSTANTIAL testing. Almost certainly you will want your
program to be adaptive rather than wiring in numbers that you picked out
of a hat.
First, and foremost, keep firmly in mind that a single download is ALWAYS
faster than multiple partial downloads. ALWAYS do the single download if
there is any way you can get it past the user.
If the need is to make the user think that the application has not "frozen
up", consider doing the download in background.
Now, given that we have some cases where you MUST break up the download,
here's what you should do:
Start with defining the bandwidths that you can reasonably expect to
encounter. For most purposes, something like 32 kbps is probably the
worst case.
Also, you ought to put a high cost on RTTs; at least 1/4 second but I
recommend closer to 1/2 second.
Oh yeah, also beware the occasional RTT From Hell when your TCP session
goes into retransmission; this will really bite you in the ass in any kind
of radio (Wi-Fi, mobile phone, etc.). We're talking 2-digit second
delays here.
Suppose you have 1000 UIDs to fetch. On our 32 kbps link, that will take
about 2 seconds as a single download. Now, if we do it 25 at a time, we
now find ourselves undergoing a task that may take 42 seconds! Perhaps it
is better to do 500 at a time.
Now consider the best case. We'll exclude wired broadband on the grounds
that we assume that we will always do single download in that case.
Wi-Fi and mobile phones (especially 3G) are much faster than 32 kbps, yet
they still occasionally get the RTT From Hell.
Your calculations become quite different now. There really is no single
value or rule that works in both the dialup and the Wi-Fi/mobile phone
cases. You have to have multiple tuning for both.
This isn't quite rocket science, but it's not seat-of-the-pants either.
I have seen entirely too many clients killed by RTTs from badly chosen
download strategies, only to hear the developer blame IMAP instead of his
poor use of IMAP.
-- Mark --
http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.
_______________________________________________
Imap-uw mailing list
Imap-uw@u.washington.edu
http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
