When discussing GV's Nightly/Beta/Stable release trains with the Focus
team, I created a doc [1] comparing the release cadences of GV, Focus,
FxR, Fenix, and A-C. They are all different and range from 1 week (A-C)
to 12-18 weeks (GV), so there will be inevitable schedule headaches.
At the end of the day, the app teams are responsible for deciding when
to update their libraries. The GV or A-C teams only have so much say
over which specific GV or A-C version gets shipped in any particular
Focus or Fenix version. :)
[1]
https://docs.google.com/document/d/1qy0ViJ-TeheS6f1TFG6iEQt7CWxFzzb5nZ2lE0FoNLM/edit
On 2018-10-02 3:27 PM, Michael Comella wrote:
One last question: do we know how other organizations solve these
types of central library problems? Though perhaps the conditions are
different enough that their experience is not relevant.
---
Tangent: I'd like to point out that *things will only get better from
here!* ๐ The components team has been in the process of
/reimplementing/ /the core functionality of our apps /while the
product teams are creating complex feature releases (Focus w/ GV and
Fire TV redesign). It's incredible we haven't had more bugs, actually. :P
Future releases may not be so risky and when the core functionality is
completed, upgrade timelines will be less demanding and bugs will be
less critical to fix quickly.
Also, perhaps more importantly,*our teams are showing a commitment to
learning and iterating to improve our processes *(thank you again,
Stefan, for starting this thread!). Thanks for all your hard work
everyone and I look forward to seeing where we go next! ๐
- Mike
On Tue, Oct 2, 2018 at 3:07 PM Michael Comella <[email protected]
<mailto:[email protected]>> wrote:
Thanks for starting this discussion, Stefan! I think it's helpful
to better understand everyone's problems so we can come to a
shared process that benefits us all!
To take a step back, on Fire TV some of the issues we've faced
with components integration are:
1) The way the product and components release schedules happen to
align *makes us find and fix bugs/design issues later than we'd
like.* The product sprints/releases are every two weeks, we code
freeze on the second Wednesday, and finish our build end-of-day
Friday. afaik, components releases sometime between Wednesday and
Friday which means we're sometimes updating the potentially
unstable components [1] after code freeze, creating a lot of
stress on both teams to fix bugs or design issues by the end of
the week.
Some solutions, with varying trade-offs:
- Upgrade components once a sprint (during week one): I think this
is what Susheel proposes by pinning and backporting dependencies
- *Desynchronize the two release schedules,* e.g. release
components earlier in the week
- Release components more often so we can test earlier before an
official components release: this is what Stefan proposes
It's important not to fall too far behind components due to the
additional engineering overhead [2] so I'd rather upgrade the
components frequently, if possible, removing option #1. *#2 seems
like a great way to keep our products more stable* but it may not
be logistically feasible (e.g. does sprint planning align?). I
think #3 is also a good option to improve stability but we may
still find the final upgrade from final RC to release creates bugs.
2) *Components updates unexpectedly break existing integrations*
(e.g. new Settings
<https://github.com/mozilla-mobile/firefox-tv/issues/1204>),
adding unplanned work to our sprints.
Some solutions:
- Explicit notifications about potentially breaking changes early:
this can let us plan for the fixes in sprint planning
- Only upgrade components early in the sprint to find and fix such
issues early
As before, I'd prefer to upgrade components as often as we can
while maintaining stability so I'd go with #1. I want to see more
false positives! :) A solution to the first problem will also help.
Chris' later email points out semantic versioning but I think some
our breakage often comes from 1) incorrect integrations, 2) design
issues, 3) missing functionality in components that replaces
WebView-based functionality, and 4) new components code breaking
existing work-arounds: semantic versioning wouldn't address this.
3) *Patching previous releases seems to have a lot of overhead.*
Sometimes a product is forced to pin their versions for stability
or lack of engineering time to integrate breaking changes (see
above) but it's seemed difficult to quickly backport bug fixes.
From this thread, it sounds like the components team is addressing
this though. ๐ Also, finding bugs sooner (the point of this
thread!) would make /quick/ backports less necessary.
---
Finally, wrt snapshot vs. RC, I think it's better to have
explicitly pinned versions so we're never in a situation where
behavior is not reproducible. We can also release with an RC when
necessary.
Hope this helps!
- Mike
[1]: They're stable against the components test suite but that
doesn't mean they integrate well with our app design or that
they're bug free when used in our product (e.g. Fire OS keyboard
bugs).
[2]: Additional engineering overhead comes from the time spent
backporting and, when upgrading to the latest version, finding and
fixing bugs/design issues that were fixed some time ago in
components so it's harder to track down.
On Tue, Oct 2, 2018 at 11:31 AM Stefan Arentz <[email protected]
<mailto:[email protected]>> wrote:
On Tue, Oct 2, 2018 at 1:39 PM Susheel Daswani
<[email protected] <mailto:[email protected]>> wrote:
Yes Stefan, I don't think the snapshots and rc releases
are super useful /yet/. We do use them with GV but that's
really just because GV development is so active and the
engine is so core that updating it more frequently
overrides the risk. But being on the bleeding edge does
have it's issues (e.g., yesterday we had to downgrade GV
because it was too bleeding edge).
I think what we will recommend to all teams is actually this:
- Always develop against components -SNAPSHOT so that you stay
up to date. Snapshot is the place where you will get the
changes, bug fixes and features that you have requested. And
you will get them real-time as those changes land. Using
snapshots is the best way to work together with us because not
only will you get the things you need soon, you will also find
issues sooner which means we get to a stable release sooner,
which means less chance of point releases.
- When your project reaches code freeze or feature complete,
that is the moment you stop using snapshots and pin to a
stable version. In an ideal world the timing is such that our
calendars are in sync enough that you can just pin the
upcoming release. But if they are not, you can tell us the
point in time when you would like to pin components and we can
make a release happen on that date. (Note that this does not
scale well beyond one or two products, it is simpler to take
our release dates into account and alignย schedules.)
The point releases are to handle unexpected (aren't they
always) critical bugs, they do not exist to accommodate a
development schedule. However, they can happen after you
pinned to a specific version or even after you shipped. But I
think that with a good schedule and good testing, these
releases can mostly be avoided.
ย S.
--
You received this message because you are subscribed to the
Google Groups "Android Product Team (APT)" group.
To unsubscribe from this group and stop receiving emails from
it, send an email to
[email protected]
<mailto:[email protected]>.
To post to this group, send email to
[email protected]
<mailto:[email protected]>.
To view this discussion on the web visit
https://groups.google.com/a/mozilla.com/d/msgid/android-product-team/CAKyLfD4OXSfV%2Bf-4AmWALEJtKT%2BhwZoOp1F%3D%2BFh%3DY%3D2LhMR9CQ%40mail.gmail.com
<https://groups.google.com/a/mozilla.com/d/msgid/android-product-team/CAKyLfD4OXSfV%2Bf-4AmWALEJtKT%2BhwZoOp1F%3D%2BFh%3DY%3D2LhMR9CQ%40mail.gmail.com?utm_medium=email&utm_source=footer>.
--
You received this message because you are subscribed to the Google
Groups "Android Product Team (APT)" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to [email protected]
<mailto:[email protected]>.
To post to this group, send email to [email protected]
<mailto:[email protected]>.
To view this discussion on the web visit
https://groups.google.com/a/mozilla.com/d/msgid/android-product-team/CABupmeB2rPT7m3jOZQvO-OnO-YoWZFLBoqgM8DSDsZwZxAtjEg%40mail.gmail.com
<https://groups.google.com/a/mozilla.com/d/msgid/android-product-team/CABupmeB2rPT7m3jOZQvO-OnO-YoWZFLBoqgM8DSDsZwZxAtjEg%40mail.gmail.com?utm_medium=email&utm_source=footer>.
_______________________________________________
Sync-dev mailing list
[email protected]
https://mail.mozilla.org/listinfo/sync-dev
