Hi
My 2p
dbox plans, could be implemented for v1.2:
Interesting to see this as production ready product. Could be very
interesting if it adds performance and new features.
v2.0 master/config rewrite
--------------------------
It's mostly working already. One of the larger problems left is how to
handle plugin settings. Currently Dovecot just passes everything in plugin
{} section to mail processes without doing any checks.
Isn't one of the simplest things still to build a "hash" or "dictionary"
(pick your term) of all the things in the config file and then let the
relevant chunk of code figure out what to do with it?
You need some kind of API to get out the bunch of possible options (eg
see Xine/mplayer and how it lists all the default keymappings so that
you can easily build a custom set). But after that you should really be
able to stick in anything you like and the code just parses it and uses
sensible defaults...
If config is shared then it might be nice to break out the config parser
into a separate lib so that it's easily shared around - in particular
default configs
What about it the default configs (and hence config structure) are
provided with each plugin in a simple text file? Main app has a way to
scoop all these together if someone needs a base template config file?
Index file optimizations
Nice to reduce admin knowhow wherever possible. ie reduce the number of
corner cases where files can grow uncontrollably. This is important
when converting someone from MS Exchange to dovecot...
Proxying
--------
- These could be implemented to v1.2.
- Log in normally (no proxying) if destination IP is the server itself.
Personally I think the two main use cases here are:
1) Set of frontend servers feeding to dedicated backend servers. Simple
passthrough case
2) Smaller setup has mixed frontend/backend servers (also useful to some
extent when upgrading from some old setup with another brand of IMAP
server). Basically the frontend server should notice if it's also the
final destination and skip the proxying step, otherwise it behaves just
like step 1) above
Idea 2) above is problematic in the case of mixed backend server types,
eg I was upgrading from Courier to Dovecot. I don't think it's
currently within scope to do anything too clever here - there are other
proxy servers which can be looked at if necessary for really complex
setups. I fixed it by using the CAPABILITIES option and limiting to the
min set available on both servers - can't really see a better fix anyway
because clients often ask for CAPA before even logging in...
So lets keep this simple and see if anyone shouts for anything more
complicated before implementing it... Webmail optimisation is a far
more common request than some of the advanced stuff that you were
suggesting I should think?
Adding DNS lookups to the current proxy process would likely help some
LDAP users I think?
Replication
I'm really up for this. However, your plans all seem to involve online
sync - I need a very low bandwidth dialup sync...
Much more interested in features like
- only syncing certain mailboxes, or prioritising the order at least
- Perhaps only syncing the main mime structure and the delayed
attachments sync (really nice if could be done cleanly!!)
- Completely multi-master, eg customer walks to remote location, plugs
in and works seamlessly - walks back to main office and continues
working there. Server figures out how to combine changes later
- Really, really low bandwidth usage
You mention some problems like unique ID's. This is actually pretty
simple - just use some GUID type process. All you need is something
guaranteed unique on each server, combinations of time, something unique
per server and some randomness get you there. Discount anything which
is some kind of counter because it will break eventually.
- Support for manual resync of all mailboxes, in case something was done
outside Dovecot.
Definitely want this as robust as possible in case of corner cases where
some change gets past the replication routine
- Conflict resolution.
It's better that you get a duplicated message than something is lost.
Most usage patterns for most email are: read/delete, or read/file.
Optimise for those
- Synchronous expunges also? Maybe optional. The main reason for this is that
it wouldn't be nice to break IMAP state by having expunged messages come
back.
Expunges need to be async and tracked. Very likely we will see people
delete messages from two servers (because they moved server and didn't
see the delete sync across yet.
Also need to watch for clients copying up deleted messages...?
4. Multi-master replication:
- Almost everything is already in place. The main problem here is IMAP UID
allocation. They must be allocated incrementally, so each mailbox must
have a globally increasing UID counter. The usage goes like:
Don't see why? Use some kind of GUID and ensure that there are never
any collisions right from step one?
Study Lotus Notes on this. Absolutely fantastic sync. You can create
any kind of loops you want, shake it all around bring servers up in any
order, sync clients to random servers and it all seems to work pretty
well. This is what we really want for this
5. Automation:
- Automatically move/replicate users between servers when needed. This will
probably be implemented separately long after 4.
Don't see why this is a lower priority? Can I suggest that you *start*
with this and go back to do the normal replication afterwards?
The moving between servers will have wider usage I should think? It's
also clearly a synchronous operation and can be implemented initially as
a completely special case, changed later to use any replication code.
I think this code will initially be external to the main server, and I
think a huge amount will probably be learned about the requirements and
problems that will occur by implementing this stuff first. It doesn't
even need to be that efficient initially...
Actually this same code seems more likely to develop into a more general
demon:
- Adjust filters per user
- Adjust permissions on shared folders
- Other admin operations per mailbox (forced purge, cleanup Trash, etc)
The replication processes would be ugly to implement for v1.x master, so
this pretty much depends on v2.0 master rewrite.
OK, but rewrites are bad, incremental changes are always preferred...
Can I also add to your TODO list:
- Lemonade Profile!
- Enhancements to the Expire plugin making it easier to enforce a quota,
eg trimming Trash initially, then Sent Items, then old stuff in other
folders, etc, etc
In particular push email is all the rage right now and not having push
features and not having a "Blackberry" plugin is really contributing to
the rise of Exchange and other options right now. Please lets get open
IMAP servers back on the map by giving users push type features. OK,
it's not easy to figure out what to do right now, but it's a bit chicken
and egg - until there is some rudimentry support in IMAP servers then
the networks won't react and help, lets get something in no matter how
rubbish and see how it develops from there (make it flexible so that
people with crazy ideas can extend it...)
Good luck
Ed W
