On Tue, Oct 2, 2018 at 6:07 PM Michael Comella <mcome...@mozilla.com> wrote:
> Thanks for starting this discussion, Stefan! I think it's helpful to > better understand everyone's problems so we can come to a shared process > that benefits us all! > > To take a step back, on Fire TV some of the issues we've faced with > components integration are: > > 1) The way the product and components release schedules happen to align *makes > us find and fix bugs/design issues later than we'd like.* The product > sprints/releases are every two weeks, we code freeze on the second > Wednesday, and finish our build end-of-day Friday. afaik, components > releases sometime between Wednesday and Friday which means we're sometimes > updating the potentially unstable components [1] after code freeze, > creating a lot of stress on both teams to fix bugs or design issues by the > end of the week. > Well, don't upgrade then at that point in time. If there is no reason then do not upgrade? > Some solutions, with varying trade-offs: > - Upgrade components once a sprint (during week one): I think this is what > Susheel proposes by pinning and backporting dependencies > - *Desynchronize the two release schedules,* e.g. release components > earlier in the week > - Release components more often so we can test earlier before an official > components release: this is what Stefan proposes > > It's important not to fall too far behind components due to the additional > engineering overhead [2] so I'd rather upgrade the components frequently, > if possible, removing option #1. *#2 seems like a great way to keep our > products more stable* but it may not be logistically feasible (e.g. does > sprint planning align?). I think #3 is also a good option to improve > stability but we may still find the final upgrade from final RC to release > creates bugs. > I think you are in the best position to decide what the best timing is for your specific project. I hope that snapshots and more (pinnable) RC builds will allow teams to pick the right mix. We are not trying to mandate anything, just to give enough options to make things work for teams. The problem with #1 is that this is likely not how things work in practice. I assume that you are going to ask us to make changes and to fix bugs, or even submit changes yourself. And pulling those in only once in a two week sprint probably makes no sense then. If you ask for a new API in the Browser Engine, why would you wait two weeks before you take that change? I assume you want to pick up those changes asap so that you can write the application code against them. I think what you need to consider is that the first couple of weeks (or sprints) in a release schedule are probably a really good moment to follow snapshot builds. Make sure that we are aware of all the changes that you need and we can plan those to happen in that period. Then when that work has been done, you can pin to a release and in the last weeks of your sprint you can focus on stabilization and testing. It is not rare that things are more unstable in the beginning of a release and become more and more stable near the end. > 2) *Components updates unexpectedly break existing integrations* (e.g. new > Settings <https://github.com/mozilla-mobile/firefox-tv/issues/1204>), > adding unplanned work to our sprints. > > Some solutions: > - Explicit notifications about potentially breaking changes early: this > can let us plan for the fixes in sprint planning > - Only upgrade components early in the sprint to find and fix such issues > early > > As before, I'd prefer to upgrade components as often as we can while > maintaining stability so I'd go with #1. I want to see more false > positives! :) A solution to the first problem will also help. > Assuming you talk about API changes: we try hard to prevent breaking changes. In the unfortunate case when it is needed we write about it in the release notes. If you are uncomfortable about upgrading, ask us. We can always make time to help with that. One of the reasons to use snapshots would defintiely be to find problems early. The components have pretty decent test coverage, but I don't think we can guarantee being bug free. Even if thing work well in our demo apps or in any of the 5 products that now use components, there is always a change that something unexpected happens in a product. I have no good answer to this. Other than, it may have to be a shared responsibility between APT/QA/ACT to ensure a level of quality. One thing we did recently in Focus may be worth thinking about: we had a PR with a large refactor in Focus to integrate with the browser-session component. Because of the size of this PR, we asked QA to regression test a build coming from the PR branch. This uncovered a number of issues that we then dealt with before the patch landed. You could consider doing the same thing when you pull in a components update; do it on a PR branch and have QA test it. Chris' later email points out semantic versioning but I think some our > breakage often comes from 1) incorrect integrations, 2) design issues, 3) > missing functionality in components that replaces WebView-based > functionality, and 4) new components code breaking existing work-arounds: > semantic versioning wouldn't address this. > > 3) *Patching previous releases seems to have a lot of overhead.* > Sometimes a product is forced to pin their versions for stability or lack > of engineering time to integrate breaking changes (see above) but it's > seemed difficult to quickly backport bug fixes. From this thread, it sounds > like the components team is addressing this though. 👍 Also, finding bugs > sooner (the point of this thread!) would make *quick* backports less > necessary. > We are addressing our ability to more easily publish bugfix releases. But whether a bug can actually be backported will have to be decided on a bug-by-bug basis. It is not really a given that everything is easily backported or fixable in an older release. There may be a case where the answer is: this is better fixed by upgrading. This is a bit theoretical, but I do want to set expectations. S.
_______________________________________________ Sync-dev mailing list Sync-dev@mozilla.org https://mail.mozilla.org/listinfo/sync-dev
