My thanks to everyone who had commented on this so far. I’m replying to your questions and comments in this email so that we don’t have a threading catastrophe.
First, several of you mentioned the typo where I wrote qa-verify when the flag is qe-verify. The policy document has the correct flag name. Second, this is not process for the sake of process, but so that Program and Release Management, PI, and engineering leadership can quickly assess what is happening with features. Please note that Program / Release Management did not come up with this policy on our own. The Firefox engineering teams were simultaneously looking to create a similar policy to further alleviate releases where a feature is inadvertently not tested or enabled. A recent item was the Retained Display list not be turned on. There’s also some question about what qe-verify means. It can be requested by anyone with editbugs and granted by anyone with editbugs. And `+` does not mean the bug is verified, but that QA will verify the the fix and set the bug’s resolution to VERIFIED. I’ve updated that in the document (at https://github.com/mozilla/bug-handling/commit/9b461160df743262aa6ecb7e6a033872ce393ff1 ). Liz Henry points out we should be clear in the policy what VERIFIED means here for testing and sign-off. :mossop asked about the Trello board, it’s at https://trello.com/b/8k1hT2vh/firefox (if you need access, please ask :elan) and we use the board to track features of note so that we know to communicate them through release notes and other channels. Since we’re using a flag to indicate features behind a pref, we can use queries to indicate features that need to go on that board. Nick Alexander mentioned that features which interact with the OS, mobile OSes (Android and iOS) are all or nothing. This policy does not mean you must release behind a pref. But if you are releasing a feature that depends on a change to your OS, the bug for the feature must note that. That sort of change may not be atomic in nature, so early warning and notification of Project and Release Management, and PI is important. Jared Wein asked in the thread; MattN, and Mark Banner asked in Issues ( https://github.com/mozilla/bug-handling/issues/4, and https://github.com/mozilla/bug-handling/issues/4) about feature work involving large numbers of bugs associated with a [meta] bug. Mark’s suggestion where the flag is associated with the meta bug, but not the dependent bugs is most likely the way to go. Ritu Kothari and D. Baron asked about managing features across release trains. The behind-pref may need to be a status flag instead of a simple flag since the target version field only indicates when a bug landed in M-C and bugs supporting a feature may land across multiple releases. I’ve started a discussion of this at https://github.com/mozilla/bug-handling/issues/5. Several of you note that we’ve overloaded preferences. And perhaps we should think about releasing features under a feature flag system. I think that would afford us a way to move features toggling to the pref panes (even if we keep feature flags in a namespaced area in prefs.) Finally, I need to talk more with Shield/Normandy about this and make sure I’m not sabotaging you. Thank you all for your feedback and for helping us ship high quality code. — Emma On Wed, May 2, 2018 at 4:57 PM Emma Humphries <e...@mozilla.com> wrote: > Hello, > > We control enabling many features and changes to Firefox using preferences. > > Program and Release management as well as PI need a better view of this. > > We've written a new policy which you can read on our nascent bug-handling > mini-site: > > https://github.com/mozilla/bug-handling/blob/master/policy/feature-flags.md > > To summarize, when you are releasing a feature that "rides behind a flag", > on the bug for the feature: > > * set the behind-pref flag to + > * set the qa-verify flag to ? > * note the bug in the Firefox Feature Trello board > > We expect qa-verify to be set to + before enabling prefs on features. > > We'll be going over this at two upcoming meetings, with Reese's and JoeH's > teams. > > There are two, known open questions to resolve on the policy: > > * Features developed over multiple releases with individual patches not > verifiable by external testing (passing unit tests, but not integration > tests) such as Hello and WebRTC > * Features are often turned on in Nightly, ignoring prefs using compiler > directives, and kept off in Beta and Release. Is this the right thing to > do, or should we be flipping prefs from on to off when going from Nightly > to Beta and forwards? > > The bug handling documentation is a GitHub repo to enable web publishing, > please follow up there, using Issues and PRs, rather than to this email. > > I want to acknowledge and thank: Liz Henry, Ritu Kothari, Resse Morris, > Erin Lancaster, Ryan VM, Thomas Elin, and Adam Roach for their comments and > feedback on this. > > Thank you! > > -- Emma Humphries > _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
