On 7/30/13 4:22 AM, Andreas Gal wrote: > Also, as we discussed earlier, CouchDB's replication algorithm is not > a perfect fit for our star-shaped nodes. Its meant for a more > interconnected graph.
Yeah, it's kind of overkill for a star topology. We anticipate other problems with relying on its replication algorithm though, our analysis suggests it won't provide atomic multi-key updates. If you wait long enough, maybe with some out-of-band pausing and notification, then you can get any one node to exactly match the state of a different node. But if you can read or write at random times, then the target of the replication (and anyone who reads from it) will observe intermediate states that have some, but not all, of the new records. On top of that, you get those Couch-visible "_conflicts" if you allow writes to different nodes and then replicate those changes into the same place. We're generally hoping to have data-representations that don't require atomic multi-key updates, but we're not sure we can avoid them. We can achieve such updates on the client->server upstream path by using a proxy in front of the CouchDB server (which we'll probably have anyways to handle our auth scheme), at the cost of using a different write API to the proxy, but my hunch is that it won't be too hard to achieve. However, I think we *could* use CouchDB's built-in replication mechanism as an easy server backup tool. For each collection, the client reads-from/writes-to a single server node, then we replicate from that to a different data-center periodically. If we fire off the replication process in-between client writes, and make sure it finishes before the next write, then we should wind up with a coherent copy of the data in the second datacenter. Then if we lose the first datacenter, we can manually switch to the backup. I think this fits in with what mmayo said last week about manual recovery from major failures being ok. > Also, if we implement our own replication mechanism, what is the > advantage of sticking with the CouchDB wire protocol? Its actually > rather clumsy and inefficient (see proposed jsondiff delta > compression). I am not arguing for using something else than CouchDB. > I am merely asking why you think it makes sense to stick with the wire > protocol but abandon the higher level semantics of CouchDB. Are we comparing 06-Queue-Sync-CouchDB against embedding a CouchDB-ish DB in the browser and telling it to "POST /_replicate" to the server? If so: * "POST /_replicate" doesn't give us control over batching/pagination, collation, or rate-limiting, and exposes the "partial-read/write" problem that rnewman has described as the source of the majority of Sync1.0's failure modes. * We'd also have to proxy its HTTP calls (POST /_bulk_docs and GET /_changes) with something that implements our auth scheme The biggest advantage of sticking with the CouchDB wire protocol is that we get to use existing CouchDB implementations on the storage server. We'll probably have a proxy in front of it, but that'll be pretty thin. Deviating from the existing protocol means we have to write the code on both ends of our new wire. Our proposed scheme only has us writing new code on the client side. I think using /_bulk_docs and /_changes is pretty efficient if the data model puts each item (bookmark, password, etc) in a separate CouchDB document. There's clearly a spectrum here: * one-document DB, with one giant JSON blob containing all e.g. bookmarks. (I think https://wiki.mozilla.org/User:Gal/SyncDataModel proposes this, at least for bookmarks). This could be compressed and encrypted, and would be pretty dense. But modifications are either horribly inefficient (replacing the entire document for a one-byte change), or require a custom API to apply diffs (which would only work if the record were unencrypted and uncompressed, and I think we've all agreed that we need a solution that works when they're encrypted). Any two changes (even to unrelated bookmarks) will conflict and need merging. * one-document-per-bookmark (what I'm proposing). Each record is uncompressed (not much point) and encrypted. Data-size grows a bit (we pay the per-document overhead on each record). Modifications are fairly efficient (although changing a bookmark title re-uploads the URL too). Conflicts only happen when we get simultaneous modifications of two fields of the same entry. * one-document-per-field. Modifications are super-efficient (only one field is touched), and there are basically no conflicts. But overall data size grows significantly (we pay overhead on every field). And encrypting each field separately reveals a lot more information (and is a lot more malleable), weakening our security claims. For reference, our design assumes a data model that probably looks like: * bookmarks: one bookmark per doc * passwords: one password per doc * history: one history list per doc (one URL, list of visits) * open-tabs: one DB collection per device, one tab URL per doc Hope that was useful! cheers, -Brian _______________________________________________ Sync-dev mailing list [email protected] https://mail.mozilla.org/listinfo/sync-dev
