On Jul 24, 2013, at 12:26 PM, Mark Finkle <[email protected]> wrote:
> I do have a question, mainly semantics: You talk about using the Dropbox File > API to handle syncing independent records, but my first reaction would be to > use the Dropbox Storage API, which seems tailor made for records. What's your > rationale for using File API over the Storage API? > Let's be consistent on language first: Core API (which I called the File API): https://www.dropbox.com/developers/core Datastore API (which I think you called the Storage API) https://www.dropbox.com/developers/datastore One property that we probably want out of these APIs is a "compare-and-swap"-type thing. Meaning, you can say something like "update the this record to x if the revision number is y". You can do this in the CouchDB API: http://wiki.apache.org/couchdb/HTTP_Document_API#PUT and Dropbox Core API, if you store each record as a file: https://www.dropbox.com/developers/core/docs#files_put It wasn't clear to how to do this in the Dropbox Datastore API. They don't offer a REST API, only client APIs, and browsing through the uglified JS source, ti seems like it was just built on the Core API anyway. So maybe Dropbox is doing some magic under the covers. They really would prefer you update fields individually, and then conflicts like "I changed the email, you changed the phone number" can be automatically resolved, but they pretty much throw up their hands on conflicts like "I changed the email, you changed email". Since all our records are encrypted, then it's like "I changed the encrypted blob, you changed the encrypted blob" and from my understanding, the Datastore API will resolve that in a manner of its own choosing. They have a *very* simplistic way to put in resolution rules (setResolutionRule in https://www.dropbox.com/developers/datastore/docs/js#Dropbox.Datastore.Table), but I don't that will work for us. I could be wrong, but I think we'd need to use the Core API. -chris > Lastly, I worry about any local shadow data storage, mainly due to space > considerations on mobile devices. That said, we need real prototypes to get > an idea of just how much space would be consumed. It's never free, but I > think we need to find a compromise. > > In preparation of our design review, I spent a good part of yesterday trying > to digest the pros and cons of Dropbox API, CouchDB API, and TreeSync (which > I'm frankly still a bit fuzzy about). > > In short, can we get a Triple Win, where can benefit from the pros of all > three of these options? > > Observation 1: The CouchDB API or the Dropbox File API is likely good enough > for syncing independent (or even mostly independent) records (e..g, history, > passwords, tabs). Future stuff like calendar events, contacts, reading list, > etc. might fall in this category too. > > Observation 2: Bookmarks are hairy because there is structure, and changes > should probably be more transactional than what the CouchDB and Dropbox APIs > provide. We might be able to alter the Couch API slightly or work within it > to address these issues. It probably wouldn't be as good as a more tailored > solution, but it might be good enough! Weird things might still happen, but > we could accept it, particularly if users are moving away from maintaining > structured bookmarks. The upside is that we could potentially move quickly. > > Observation 3: It would be nice to have clean API boundaries and composable > subsystems. For example, it would be nice if we could have a simple API > between the content/data provider (e.g., the history component) and the sync > mediator (the thing responsible for talking to the storage server). This > could allow us to change (or allow the user to choose!) a storage provider > without having to make substantial changes to the content providers. > Likewise, content providers should handle their own merge conflicts. I would > also like new content providers to be able to make substantial progress on > integrating with sync without a lot of hand-holding from experts. Many of you > are aware that I like the idea of the Syncable Service API in Chromium, which > has similar goals. Here's a nice talk on it: > https://docs.google.com/viewer?a=v&pid=sites&srcid=Y2hyb21pdW0ub3JnfGRldnxneDo2MzU1NDEwZTA1NTUwNzlk > > Triple Win?: Most datatypes would be fine with Couch or Dropbox. I propose we > consider a structure where the sync/wire protocol could be either the CouchDB > or Dropbox API and *data types that need more (e.g., bookmarks)* can layer > additional mechanisms on top of that. For example, TreeSync could run on top > of an abstraction backed by either the CouchDB API or Dropbox API (I think > they are close enough that we can pull that off). Brian and I brainstormed > how one might do this yesterday, and the idea has legs. > > I call this a potential Triple Win, because if it works, then we could get > the "already debugged, already there" wins from both Couch and Dropbox, we > get the integrity win of TreeSync, and we get the Dropbox > offload-some-server-storage win if that somehow makes sense. > > In this world, I don't envision needing a local CouchDB shadow copy of data > (although it isn't ruled out). Instead, this sync mediator component serves a > CouchDB or Dropbox proxy, which may do some caching of data for performance > purposes. > > -chris > > > _______________________________________________ > Sync-dev mailing list > [email protected] > https://mail.mozilla.org/listinfo/sync-dev >
_______________________________________________ Sync-dev mailing list [email protected] https://mail.mozilla.org/listinfo/sync-dev
