On 17 September 2012 13:57, Timo Sirainen <[email protected]> wrote: > On 14.9.2012, at 18.16, Damien Churchill wrote: > >> On 14 September 2012 15:59, Timo Sirainen <[email protected]> wrote: >>> On 14.9.2012, at 17.46, Damien Churchill wrote: >>> >>>> I was wondering what would be entailed in modifying dovecot 2.2 to >>>> support storing mail in an object store. I've seen a few mails dotted >>>> around in the ML history about supporting such a thing and seen it's >>>> basically dependant upon some changes in lib-storage to support >>>> writing messages without locking. Is this still the case? >>> >>> I've a whole new design for it and I was planning on implementing it for >>> v2.2. Do you want to help coding it? :) Which storage would you want to use? >>> >> >> That's good to hear :) I've been evaluating RADOS as an object store, >> which is similar to S3. Although any distributed storage would be >> great. I'd be more than happy to help code it! > > I think I'll first have to get started with it to see if there are some parts > that are easy to give to you. First I'll at least need to do some refactoring > to dbox code and lib-fs code. I'm planning on making the generic parts of it > be part of Dovecot releases, but I haven't yet fully decided which backends > should be.. > >>> The generic idea is: >>> - only one server accesses one user simultaneously >>> - index files are copied from object storage to local filesystem and >>> accessed there, once in a while uploaded back to object storage >>> - if user is accessed from two servers because of some bug/split >>> brain/something, the changes are merged using dsync >>> - support high latency: asynchronous reads/writes. prefetch mail bodies. >> >> I'm assuming that the director would be used in order to distribute >> connections to the same server, so it's only within a local instance >> of dovecot you'd need to be aware of what currently has a connection >> open for that user? > > Right. Probably some new process that can do the work of > downloading/uploading/deleting index files as needed. That's actually a > clearly separate task that you could do? :) >
Sounds good! I'll spend some time digging through the source code getting familiar. >> How are you planning on handling the situation where say node X dies >> and hasn't uploaded the latest index file? Would that result in >> missing messages from the mailbox when accessed by another node, or is >> the local index intended to be more of a write-through cache? > > No messages get ever lost. Recent flag changes and expunges may get lost, at > least until the original node comes back up and dsync merges the changes. > Idea was that when downloading index a flag on the object storage is set for > the user that it's being accessed, and removed after the user is disconnected > and index is uploaded back. If index downloader already sees that the flag is > set it will run some kind of a recovery process to find any messages that > were uploaded but not indexed. (The message bodies are always immediately > uploaded to object storage.) Part of me thinks making this configurable might be a good idea depending upon what the installation is trying to achieve. Since the recovery process will need to be implemented regardless allowing the user to configure dovecot to perform a write to both the local and object index when a flag or something else is modified. Another thought occurs to me, when using the LDA how will that be able to update the index upon delivery of a new message if another node is currently accessing the mailbox?
