Hi
The updated design #2 should address this. The mailbox synchronization
step works pretty much the same as QRESYNC.
Thanks will look through that
a) Add a FETCH-SMALL command that drops large attachements and somehow
remembers this so that they could later be downloaded again when there's
more bandwidth or user requests it.
Agreed - it's tricky.
Not 100% sure myself how to handle this yet. I think it requires some
extensions quite a long way outside of the normal IMAP process, but it's
probably a very nice feature for those that need it.
Using IMAP protocol for replication has at least two disadvantages:
1) It's a bit too chatty, wasting bandwidth on replies the replication
isn't interested in.
Compression eliminates a huge amount of traffic (I typically see 12:1 or
better). Also pipelining commands eliminates much of the disadvantage
of the chatty behaviour (lets assume >1sec latency on most dialup links,
so latency is definitely a killer)
I currently use a small self written proxy app which does some simple
analysis of what imap client is talking and does some prefetching via
pipelined commands to reduce latency and also sets up a compressed pipe
back to the server. Even over broadband it gives quite a significant
speedup on large folders and on dialup it gives a huge performance
boost. Nothing too clever going on though
Perhaps we could look at some optimisations like that in the first instance?
2) Sending updated flag/keyword changes can't be done in a standard way,
because it only shows the last flag+keyword state, not the changes that
were done (e.g. "\Seen" vs. "+\Seen -\Flagged").
Hmm... I guess that can only be done anyway by storing the state before
and after and figuring out the changes based on a comparison?
I do like the idea of making this more generic and hence hackable than
writing all the code into dovecot itself. Perhaps we could start with
an external proxy app at each end of the link which is external to the
imap server, ie basically start with IMAP sync. This seems easy to
knock up in teh scripting language of choice (eg imapsync) and we can
then easily hack on the protocol and choice of commands to bring the
servers into sync. I guess if some obvious bottlenecks occur then it's
simple to make the protocol across the wire slightly different and
subsequently look at how those changes could be moved into the imap
protocol itself?
The proxy app then gives a clean break to monitor stuff like changes in
flags (expecting this ot need some support from the server though to
avoid duplicating the index data?). It would potentially make it
possible to support other imap servers than dovecot, although I don't
believe that should be on the roadmap, but others may want to code that
up themselves?
So effectively start with ImapSync style app and then use knowledge of
the IMAP server and the QRESYNC stuff to make it very much more
optimised. Other imap servers then have the option to code up the
requried missing features and we have invented a standardised way to
sync two servers...
Sound any good?
Ed W
