On Jul 24, 2013, at 12:26 PM, Mark Finkle <[email protected]> wrote: > > Observation 1: The CouchDB API or the Dropbox File API is likely good enough > for syncing independent (or even mostly independent) records (e..g, history, > passwords, tabs). Future stuff like calendar events, contacts, reading list, > etc. might fall in this category too.
I did some digging into the Dropbox APIs (summarized below) that suggest it may not be quite as simple as we might hope. TL;DR: Close, but all three APIs have (different) flaws that make them not quite there. I'd love to be corrected if I'm wrong, since I think it'd be a big win to be able to use them, but I think it goes a little past overwrite-reconciliation logic questions. Toby > One 'problem' that all three systems don't address is the idea of priority. > They just give you the changes in non-user-controlled order, so you don't get > the important updates first. In an increasingly connected world, this may be > less of an issue, but it was something the client developers *really* wanted > in the early days of sync. > > The new Datastore API is pretty damn cool. If I was an app developer who > needed to build some syncable preferences into an app, I'd be super-happy > right now. From a modeling paradigm, it does pretty much exactly what we'd > want. The limits on the values aren't entirely clear, but I don't think this > would be a blocker. > > The big problem with the Datastore API is that it's really invasive. It's not > an API, it's essentially an app that you write data into and read data from > and it takes care of syncing that invisibly. In order to make it work with > the various pieces of data we work with (places, the prefs, etc), we'd have > to set up watchers on that app and effectively figure out how to sync from > that to the correct place (or replace all our internal systems with this, > which is clearly a non-starter). This seems really inefficient, and prone to > a lot of errors, as well as a lot of data duplication. > > The Sync API is closer. It lets you set up a 'filesystem' that gets synced. > It's easy to imagine that filesystem with folders for bookmarks, passwords, > forms, etc as folder containing a file for each item. You can listen to the > folder for changes and act on that. It has the same inefficiencies as the > Datastore API (duplicated data), but is a little less black-boxy, so it's a > little easier to monitor. The Sync API, however, is only available for iOS > and Android. There isn't even a Javascript client mentioned, which means > that, unless I've just totally missed a section, it's not supported for us. > > Finally, there's the raw Core API, which is back where we want to be - a set > of methods on URLs. Going through it, it's straightforward to build - it has > get/delete/post as well as a bunch of helper functions that we might (or > might not) use that you'd expect from a filesystem. Each item would be a > 'file' as translated from our helper layer, but we wouldn't need to keep it > as files from our side. Most significantly, it has a /delta function that is > effectively a vector clock - you give it your last state, and it gives you a > list of changes from that state, plus a new state. It's basically exactly > what we want… with one crucial flaw. > > That /delta function is only available at the top level. You can't get a > delta on a particular folder. We could hack it by getting everything, > filtering out stuff that wasn't in our folder, then only downloading our > part, but it also means that anything else using this API will be dealing > with the thousands of files that we generate. This is such an obvious flaw > that I have to believe it's intentional, and not a technical difficulty. > Maybe we're willing to put up with this. _______________________________________________ Sync-dev mailing list [email protected] https://mail.mozilla.org/listinfo/sync-dev
