On Fri, Sep 2, 2016 at 4:52 PM, Nicholas Alexander <[email protected]> wrote:
> On Fri, Sep 2, 2016 at 4:41 PM, Matthew N. <[email protected]> wrote: > >> Hello sync-dev, >> >> Is there documentation on best practices for making a new data type that >> will eventually be synced? I'm trying to figure out a schema and storage >> backend for Form Autofill profiles (on desktop to start) and though we're >> not implementing the sync engine initially, I would like the data to be >> ready for sync without significant re-implemention of the storage. >> > > I'm not aware of a link to such documentation. > OK, I kinda suspected that since the harder part is the engine itself. On the schema side, an obvious thing is to use a GUID (shorter ones are >> preferred IIUC) instead of an incrementing ID for the primary key. It seems >> things like tracking of deleted records and how to handle conflicts between >> the server and client can be figured out when the engine is added but >> correct me if I'm wrong? >> > > You're correct. A GUID as the primary key is preferred, since you'll need > to track a (remote) GUID anyway. > > As for best practices, the number one thing is to remember that Sync is an > object store with no good way to maintain references between objects. > Therefore, if you can keep records independent of each other, you will have > a *much* better time. As an example, history records are independent, but > bookmark records are not: bookmark folders reference bookmark items (and > vice-versa -- parents!) and no end of pain ensues trying to keep references > coherent. > Right, I see Chromium is using multiple tables and one-to-many relationships for data like names, email addresses and phone numbers but I have yet to figure out if that's something we will want. I would guess their design is optimizing for the case where you add a phone number or email address to an existing profile in which case it's just a matter of adding a row with two fields (guid, number/address). I guess it depends on how we would want to handle a conflict where two clients changed their phone numbers (added or removed) before a sync happened. Some methods for handling that would require more than just an array of strings containing phone numbers e.g. timestamps. > Concretely, that means you should prefer a JSON record per-profile rather > than a profile record with sub-item records; or perhaps you would make each > record contain a "profile name" and an item identifier, etc. > Makes sense. > Regarding the storage method, my understanding is that SQLite is used on >> mobile to provide access to the data from a separate non-Gecko sync process >> (at least on Android, not sure how things work on iOS). For desktop I would >> prefer to use a simple JSON file like logins.json since we don't need the >> features of an RDBMS but it's possible to re-use a SQLite backend between >> desktop and mobile then I'll consider it. I'm not sure if that's the case >> though since it seems like each client would write it's own native wrapper >> around the SQLite >> > > You may be able to share storage implementation code, if you back your > store with SQLite. Since you don't want to (JSON on Desktop), you won't be > able to. So Android will require its own storage implementation, and its > own Sync engine implementation. iOS will require the same. > I said I'd consider using SQLite for desktop if we can share implementation code. I guess I was hoping for a shared JS-implemented version but didn't think that would be usable on iOS, for example (at least I don't think storage-mozStorage.js was used for passwords on iOS). > Storage implementations don't have to be complicated on Android; neither > do Sync engine implementations. (Once you've decided how to reconcile > conflicts, of course!) > Sure, but it adds to the testing matrix and I would like to avoid that if possible. (I saw Passwords was moving to a Java implementation on Android though I'm >> not sure of the motivations for that TBH) in which case very little code >> would be shared. >> > > The storage layer exists, but it's not currently used. The motivation is > multi-fold. First, we see many PW storage failures on Android. That is, > the DB is corrupt, crypto failed, whatever. Second, we see concurrency > problems, performance loss, and memory waste due to the required additional > process for Gecko's SQLite compatibility. And finally, we'd get closer to > severing the Fennec front-end from Gecko if the store was independent of > Gecko. (In general, we've learned never to use Gecko if we want a good > Android experience.) > It seems like these same motivations would apply to form autofill profiles and therefore I likely wouldn't be able to share a JS-implemented storage implementation, right? > Happy to elaborate in more detail; it would help for you to post your > suggested schema and Desktop storage proposal. > Yeah, I plan to share those when I have them figured out. I wanted to hear what the list had for suggestions before writing my ideas up. Thanks!
_______________________________________________ Sync-dev mailing list [email protected] https://mail.mozilla.org/listinfo/sync-dev
