Re: looking for Cyrus mail format documentation

[John A. Tamplin]

Phil Howard wrote:

One of the needs I have is to build a two-way mail store replica.  Either
node may be delivered to, and either node may be accessed by the user but
only one at a time.  The two nodes are topologically and geographically
far apart, and bandwidth between them is to be considered costly and thus
should be not much more than the cost of actually transferring content.
If mail arrives at one, it should be replicated to the other ASAP.  If
mail is deleted at one, it should be deleted from the other ASAP.  If
mail is moved around between folders unchanged, it should be moved the
same on the other without transferring content.  Now here is the big one:
If the two nodes are unreachable between each other, changes have to be
stored in a way they can be re-syncronized when reachability is again
established.  And this may involve some changes to both and some issues
that have to be dealt with as best as possible such as noting dates of
changes (it can be assumed the two nodes are time syncronized).

This is one of needs I have.

Then I would suggest a better way of doing that than trying to figure out what changes have happened by looking at low-level data structures would be to put proxies in front of Cyrus (LMTP, IMAP, and if you use it POP). The proxies would pass the data on to the local Cyrus to do the action as well as contacting the other proxy to duplicate the work. If the other proxy is not accessible, keep a log of the work that needs to be performed (but allowing disconnected operation when the other node is not truly down will likely lead to changes that can't be automatically resolved -- better would be to have 3, run two-phase commit and only commit if you get agreement of two, but that may not be practical) and do those changes when the other proxy comes back up. The level of abstraction you want is precisely the level you get at the higher level protocols rather than having to dig through all the folders and see what has changed.

Of course replicating some things such as seen state will be quite painful, and you may need to do some hacks to keep uids unique between the machines.

--
John A. Tamplin
Unix Systems Administrator