On Wed, Oct 18, 2017 at 12:29 PM, Thom Chiovoloni <tchiovol...@mozilla.com> wrote: > Hello! > > The Sync team currently has some pieces of code in desktop Firefox that > would benefit from (more) rapid iteration. Specifically, the code pertaining > to bookmark validation and repair. > > # Overview > > Currently, a large portion of Sync users have corrupted bookmarks in their > server-side bookmark tree. There are a number of ways this could happen, > such as non-atomic syncs (the server only recently landed support for atomic > uploads across multiple requests), bugs in one or more clients, and a number > of other ways. > > For a long time this prevented iOS devices from syncing bookmarks at all, > and while iOS has some ad-hoc fixes to enable it now, corruption still can > lead to data loss, and the fact that the tree is in a corrupt state still > limits our ability to avoid further corruption. > > For reasons like these, Sync grew validation and repair code to measure and > fix the issue (the validation code came much before the repair code, and the > repair code was originally written due to iOS being blocked). The repair > code focused on a few specific issues, with the hope that the others would > get fixed automatically (there are a number of issues that imply other > issues will be present). In practice it was insufficient. > > This code currently runs on Beta and Nightly. We've been reluctant to move > it to release due to it's relatively high cost (it's fairly slow, as it > necessarily requires fetching the full server bookmarks tree). We already > run for a small subset of users, and only in cases where we know the record > count is relatively low. (Specifically, we consider running validation at > most once a day, and when we consider it there's only a 10% chance we'll > actually do it, and even then it will only happen for users with < 1000 > bookmarks). > > # Timeline > > The code is already written and lives in Sync, but needs to be factored out. > We also will need to develop mechanisms for handling it's presence or lack > thereof, etc. We don't expect this to be terribly time consuming, a month or > so seems safe (but I've been wrong before). > > I don't think we plan on most of the iteration on repair/validation to be > done until Q1 2017, so riding the release train for 58 or 59 would likely be > fine (my understanding is that it's somewhat common for system addons to > ride the release train, after which they can be iterated upon more quickly > -- I can't find documentation confirming or denying this, however).
The rest of this all seems reasonable to me, to address this bit specifically: Yes this is a common pattern for things we ship as built-in features - we have docs at http://firefox-source-docs.mozilla.org/toolkit/mozapps/extensions/addon-manager/SystemAddons.html and also on the wiki at https://wiki.mozilla.org/Firefox/Go_Faster but I am not sure if it covers this explicitly. The in-tree docs (the firefox-source-docs link) do cover the technical differences between built-in and updates, there's a lot of flexibility on how you want to ship. > > # References > > The bug tracking this work, bug 1407067, > https://bugzilla.mozilla.org/show_bug.cgi?id=1407067 > > Most of the sync code in question: > - > http://searchfox.org/mozilla-central/source/services/sync/modules/bookmark_repair.js > - > http://searchfox.org/mozilla-central/source/services/sync/modules/bookmark_validator.js > - > http://searchfox.org/mozilla-central/source/services/sync/modules/doctor.js > > Well, I hope that all makes sense. > > - Thom Chiovoloni [:tcsc] > > _______________________________________________ > Gofaster mailing list > gofas...@mozilla.org > https://mail.mozilla.org/listinfo/gofaster > _______________________________________________ Sync-dev mailing list Sync-dev@mozilla.org https://mail.mozilla.org/listinfo/sync-dev
