On 13-07-28 7:21 PM, Andreas Gal wrote:
https://wiki.mozilla.org/User:Gal/SyncDataModel
Some comments on the bookmarks section of the strawman:
* Key: "bookmarks" for desktop bookmark tree, "mobile" for mobile
bookmark folder
nit: "desktop" for desktop bookmark tree.
nit: On Android, we have several top-level roots: "mobile", "pinned",
"readinglist", "unfiled"; and on desktop, "toolbar", "menu".
* The entire bookmark tree is represented as one doc.
The tree consists of folders ({ description, title, children }),
separators ({}), and bookmarks ({ title, uri, description,
loadInSidebar, tags, keyword}). We expect dozens of bookmarks, and
many hundreds worst-case.
Dozens? Many hundreds? This expectation is reasonable for the
"mobile" folder. (Caveat: we don't have any users building out the
"mobile" folder for more than 18 months.) But per
https://id.etherpad.mozilla.org/picl-user-model, only 1/2 of our
existing Sync users have fewer than 100 bookmarks; in fact, 1/6 has
more than 10000 [1]. A monolithic bookmark document makes a lot of
syncing much easier, but I *really* don't think it scales to the
thousands of bookmarks users we see in the wild.
I think everyone is aware of the bandwidth cost, but the strawman has
a high memory and disk access costs. These can be partially mitigated
by GUID tagging and versioning individual records in the bookmark
trees.
The strawman provides no way for a client to quickly determine what
parts of the tree have changed since last sync. On some of our lower
end devices, disk access is painfully slow [2]. We probably can't
afford to write (or enumerate) the entire bookmarks tree on each
sync. Versioning records inside the tree will allow us to
selectively update the DB, and GUID tagging will let us not enumerate
the tree entirely.
The temporary storage for even a 50k encrypted-JSON ciphertext (and its
plaintext) is going to hurt on Android [3]. Since the strawman
doesn't provide structural help to the client side reconciliation, I
expect we'd need to maintain the incoming tree for the whole sync, at
the same time as requiring expensive queries to the local DB for the
"before" state. Reconciling a "moved and renamed" sub-folder will
require even more expensive queries. GUID tagging the individual
records in the bookmark tree will help with this (but not, for
example, with reconciling adding the same bookmark on two devices
independently).
Best,
Nick
[1] I wouldn't be surprised if the average Sync user has tons more
data than the average Firefox user, but I haven't seen data to that
effect.
[2] Android kills background syncs after 5 minutes. I vaguely recall
a device that couldn't write 500 bookmarks in that 5 minutes. But I
can't verify this from my IRC logs or email, so let's assume that the
world is a better place now :)
[3] For context, most of our Android devices have an 8 meg heap and
Sync is typically running with 7 - 7.5 of those 8 megs allocated.
This is based on published Android limits and observation of my and
rnewman's test devices. I imagine b2g devices are even more
constrained. We could add telemetry to better understand this.
_______________________________________________
Sync-dev mailing list
[email protected]
https://mail.mozilla.org/listinfo/sync-dev
