Re: Triage Plan for Firefox Components

[johnarsenal886]

Hi Emma,

Sorry to bother you again.

I built a tool Issue Workflow Explorer (IWE) to help better understand and 
investigate issue policies in history. The demo is available online at 
http://yuekeplus.cn/iwe/demo/mozilla. The document is at 
http://yuekeplus.cn/iwe/tutorial.

Now as I'm wondering whether this tool could really contribute to the 
community, it would be my great honor to hear any feedback or advice.

Sorry that the data is only available till 2012, and the app needs some time 
loading.

Best,
John
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[johnarsenal886]

On Friday, 14 October 2016 04:10:18 UTC+8, Emma Humphries  wrote:
> 
> 
> ​Hi John,
> 
> Thanks for your patience while I got back to you.
> 
> The triage plan we're using is
> https://wiki.mozilla.org/Bugmasters/Process/Triage.
> 
> The Platform team is also using ​
> https://wiki.mozilla.org/Platform#Bug_Triage to focus on regressions.
> 
> Several teams that develop on GitHub are using the P1-P3, P5 scheme in
> their repos and using waffle.io to aggregate across projects.
> 
> Scoreboards are:
> 
> https://emceeaich.github.io/triage-report/ and
> https://sql.telemetry.mozilla.org/dashboard/firefox-triage
> 
> My current reporting doesn't account for projects that exclusively track
> bugs in GitHub issues.
> 
> There's a mailing-list, firefox-triage-le...@mozilla.org, but it's a low
> volume list.
> 
> -- Emma

Thank you so much Emma! Really grateful for your response!
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Emma Humphries]

On Fri, Oct 7, 2016 at 2:29 AM,  wrote:

> Hi I'm someone who is studying the triage process and workflow among OSS,
> Can anyone tell me does the new triage plan discussed here get adopted and
> what is the final version? I would also love to know where and could I
> follow your discussions on policies? I can't find other topics related to
> this thread in google groups.



​Hi John,

Thanks for your patience while I got back to you.

The triage plan we're using is
https://wiki.mozilla.org/Bugmasters/Process/Triage.

The Platform team is also using ​
https://wiki.mozilla.org/Platform#Bug_Triage to focus on regressions.

Several teams that develop on GitHub are using the P1-P3, P5 scheme in
their repos and using waffle.io to aggregate across projects.

Scoreboards are:

https://emceeaich.github.io/triage-report/ and
https://sql.telemetry.mozilla.org/dashboard/firefox-triage

My current reporting doesn't account for projects that exclusively track
bugs in GitHub issues.

There's a mailing-list, firefox-triage-le...@mozilla.org, but it's a low
volume list.

-- Emma
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[johnarsenal886]

On Wednesday, 30 March 2016 04:08:45 UTC+8, Emma Humphries  wrote:
> tl;dr
> 
> In Quarter Two I'm implementing the work we’ve been doing to improve
> triage, make actionable decisions on new bugs, and prevent us from shipping
> regressions in Firefox.
> 
> Today I’m asking for feedback on the plan which is posted at:
> 
> https://docs.google.com/document/d/1FFrtS0u6gNBE1mxsGJA9JLseJ_U6tW-1NJvHMq551ko
> 
> Allowing bugs to sit around without a decision on what we will do about
> them sends the wrong message to Mozillans about how we treat bugs, how we
> value their involvement, and reduces quality.
> 
> The Firefox quality team (myself, Mike Hoye, Ryan VanderMeulen, Mark Cote,
> and Benjamin Smedberg) want to make better assertions about the quality of
> our releases by giving you tools to make clear decisions about which bugs
> must be fixed for each release (urgent) and actively tracking those bugs.
> What We Learned From The Pilot Program
> 
> During the past 6 weeks, we have prototyped and tested a triage process
> with the DOM, Hello, and Developer Tools teams.
> 
> Andrew Overholt, who participated in the pilot for the DOM team, said, “A
> consistent bug triage process can help us spread the load of watching
> incoming bugs and help avoid issues falling through the cracks."
> 
> During the pilot, the DOM team uncovered critical bugs quickly so that
> people could be assigned to them.
> 
> The pilot groups also found that the triage process needs to be fast and
> have tooling to make going through bugs fast. It’s easy to fall behind on
> triage for a component, but if you stay up to date it will take no more
> than 15 minutes a day.
> 
> You can find the bugs we triaged during the pilot by looking for whiteboard
> tags containing ‘btpp-’.
> 
> It is also important to have consistent, shared definitions for regression
> across components so triagers do not waste effort on mis-labeled bugs.
> Comments?
> 
> I am posting this plan now for comment over the next week. I intend to
> finalize the triage plan for implementation by Tuesday, April 5th. Feedback
> and questions are welcome on the document, privately via email or IRC
> (where I’m emceeaich) or on the bugmast...@mozilla.org mailing list.
> Timeline
> 
> January: finish finding component responsible parties
> 
> February: pilot review of NEW bugs with four groups of components, draft
> new process
> 
> Now: comment period for new process, finalize process
> 
> Q2: implement new process across all components involved in shipping Firefox
> Q3: all newly triaged bugs following the new process
> 
> -- Emma Humphries, Bugmaster

Hi I'm someone who is studying the triage process and workflow among OSS, Can 
anyone tell me does the new triage plan discussed here get adopted and what is 
the final version? I would also love to know where and could I follow your 
discussions on policies? I can't find other topics related to this thread in 
google groups.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Mark Côté]

On 2016-04-13 9:34 AM, Gervase Markham wrote:
> On 12/04/16 21:01, Mark Côté wrote:
>> Meant to reply to this earlier... BMO has a User Story field that sounds
>> like it does exactly what you want.  It's an editable field that keeps
>> history (admittedly not in an easy-to-read way, but that could be
>> improved).  Despite the name of the field, I've found it useful for
>> summarizing the current state of the discussion in the bug (sometimes
>> along with the "obsolete" comment tag).
> 
> If the manager of the Bugzilla dev team is 'abusing' this field in this
> way, perhaps it is indeed time for it to be relabelled - or for us to
> finally fix bug 540:
> 
> https://bugzilla.mozilla.org/show_bug.cgi?id=540
> 
> Gerv
> 

We'll seriously consider renaming this field.  I don't think this is the
same thing as editable comments, though (and we have a different idea
for approaching that).  Regardless, this is getting way off topic here.
 Unless there's more follow-up around using this field for triage
specifically, we can continue this in another thread, preferably in
tools.bmo.

Mark

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Gervase Markham]

On 12/04/16 21:01, Mark Côté wrote:
> Meant to reply to this earlier... BMO has a User Story field that sounds
> like it does exactly what you want.  It's an editable field that keeps
> history (admittedly not in an easy-to-read way, but that could be
> improved).  Despite the name of the field, I've found it useful for
> summarizing the current state of the discussion in the bug (sometimes
> along with the "obsolete" comment tag).

If the manager of the Bugzilla dev team is 'abusing' this field in this
way, perhaps it is indeed time for it to be relabelled - or for us to
finally fix bug 540:

https://bugzilla.mozilla.org/show_bug.cgi?id=540

Gerv

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Chris Peterson]

I've long thought that Bugzilla should be more like Wikipedia: the 
"front page" of the bug is editable and always up-to-date (i.e. not 
incorrect or outdated STRs), but the history and meta discussion is 
still available on a "back page".



On 4/12/16 2:19 PM, David Lawrence wrote:

I used to think it should be called "Abstract". Sort of a summarization
of the bug itself.

dkl

On 04/12/2016 05:02 PM, Emma Humphries wrote:

This is probably a field that could stand to be re-labled, as I was
blithely thinking (and I would guess others are) that it was for features,
only.

-- Emma

On Tue, Apr 12, 2016 at 1:01 PM, Mark Côté  wrote:


On 2016-04-07 2:50 AM, L. David Baron wrote:

(I'd much rather a bug report be editable text, with history
available, for answers to these or similar questions -- rather than
a stream of permanent comments.  But we seem stuck with the horrid
stream-of-comments Bugzilla format, which means I try to ignore the
comments as much as I can.  Then again, a 200 character summary is
often good enough to answer the above 5 questions.  As with the rest
of the Internet, don't read the comments.)


Meant to reply to this earlier... BMO has a User Story field that sounds
like it does exactly what you want.  It's an editable field that keeps
history (admittedly not in an easy-to-read way, but that could be
improved).  Despite the name of the field, I've found it useful for
summarizing the current state of the discussion in the bug (sometimes
along with the "obsolete" comment tag).

Mark

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Emma Humphries]

This is probably a field that could stand to be re-labled, as I was
blithely thinking (and I would guess others are) that it was for features,
only.

-- Emma

On Tue, Apr 12, 2016 at 1:01 PM, Mark Côté  wrote:

> On 2016-04-07 2:50 AM, L. David Baron wrote:
> > (I'd much rather a bug report be editable text, with history
> > available, for answers to these or similar questions -- rather than
> > a stream of permanent comments.  But we seem stuck with the horrid
> > stream-of-comments Bugzilla format, which means I try to ignore the
> > comments as much as I can.  Then again, a 200 character summary is
> > often good enough to answer the above 5 questions.  As with the rest
> > of the Internet, don't read the comments.)
>
> Meant to reply to this earlier... BMO has a User Story field that sounds
> like it does exactly what you want.  It's an editable field that keeps
> history (admittedly not in an easy-to-read way, but that could be
> improved).  Despite the name of the field, I've found it useful for
> summarizing the current state of the discussion in the bug (sometimes
> along with the "obsolete" comment tag).
>
> Mark
>
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Eric Shepherd]

This also has the effect of being somewhat confusing for the docs team,
since we may use your priority labels when planning the order in which
to work. Having a standardized system would make our lives easier and
improve the quantity and quality of the material we produce to document
y'all's work.

> I think we need to have a uniform way of identifying bugs that matter to a
> release. Having a different mechanism for each team means that we're likely
> to miss something.

-- 

Eric Shepherd
Senior Technical Writer
Mozilla Developer Network 
Blog: https://www.bitstampede.com/
Twitter: http://twitter.com/sheppy
Doodle: http://doodle.com/the.sheppy

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Mira Edorra]

My personal gmail is : anna.j.hite 

On Friday, April 8, 2016 11:53 PM, Douglas Turner  wrote:
 

 Emma,

Thanks for doing this.

I'm not sure whether something like this would work for platform engineering, 
but we'll keep an eye how things develop with Firefox and might consider it 
once we have some experience there.  I encourage you to report back here when 
Firefox starts using this system and has some lessons learned.

I also want to thank all of the people that participated in this conversation.

Doug
___
dev-planning mailing list
dev-plann...@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-planning


  
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Emma Humphries]

On Wed, Apr 6, 2016 at 6:47 PM, Emma Humphries  wrote:

>
> I'd like to finish up feedback on by the end of the working day Thursday
> the 6th (PST.)
>
> Then we'll get to work on a solid specification for the work so we can
> start implementation sometime in Q2.
>


​Okay, closing off discussion for now. If you need to follow up, contact me
on #bugmasters or email.

I'm working on a gdoc with the updated proposal.

-- Emma​
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Eric Rescorla]

Hmm.. Well, whether platform adopts this universally or not seems like a
question for
Doug and Johnny. Though, I'm sure they'll take your input seriously.

Doug, do you have any thoughts on how long an evaluation period you think is
appropriate here?

-Ekr



On Fri, Apr 8, 2016 at 3:25 PM, Emma Humphries  wrote:

> And yes, that's what I mean by Platform.
>
> Thanks.
>
> On Fri, Apr 8, 2016 at 11:24 AM, Andrew McCreight 
> wrote:
>
> > Emma can correct me if I'm wrong, but I think she is using "Firefox" in
> > the non-jargony sense of the entire thing we're shipping in Firefox,
> > including Gecko. We've been using this system for a month or so in DOM. I
> > think it has been going well. Anybody who is interested can ask Andrew
> > Overholt or I for details.
> >
> > Andrew
> >
> > On Fri, Apr 8, 2016 at 10:52 AM, Douglas Turner 
> wrote:
> >
> >> Emma,
> >>
> >> Thanks for doing this.
> >>
> >> I'm not sure whether something like this would work for platform
> >> engineering, but we'll keep an eye how things develop with Firefox and
> >> might consider it once we have some experience there.  I encourage you
> to
> >> report back here when Firefox starts using this system and has some
> lessons
> >> learned.
> >>
> >> I also want to thank all of the people that participated in this
> >> conversation.
> >>
> >> Doug
> >> ___
> >> dev-platform mailing list
> >> dev-platform@lists.mozilla.org
> >> https://lists.mozilla.org/listinfo/dev-platform
> >>
> >
> >
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Emma Humphries]

And yes, that's what I mean by Platform.

Thanks.

On Fri, Apr 8, 2016 at 11:24 AM, Andrew McCreight 
wrote:

> Emma can correct me if I'm wrong, but I think she is using "Firefox" in
> the non-jargony sense of the entire thing we're shipping in Firefox,
> including Gecko. We've been using this system for a month or so in DOM. I
> think it has been going well. Anybody who is interested can ask Andrew
> Overholt or I for details.
>
> Andrew
>
> On Fri, Apr 8, 2016 at 10:52 AM, Douglas Turner  wrote:
>
>> Emma,
>>
>> Thanks for doing this.
>>
>> I'm not sure whether something like this would work for platform
>> engineering, but we'll keep an eye how things develop with Firefox and
>> might consider it once we have some experience there.  I encourage you to
>> report back here when Firefox starts using this system and has some lessons
>> learned.
>>
>> I also want to thank all of the people that participated in this
>> conversation.
>>
>> Doug
>> ___
>> dev-platform mailing list
>> dev-platform@lists.mozilla.org
>> https://lists.mozilla.org/listinfo/dev-platform
>>
>
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Andrew McCreight]

Emma can correct me if I'm wrong, but I think she is using "Firefox" in the
non-jargony sense of the entire thing we're shipping in Firefox, including
Gecko. We've been using this system for a month or so in DOM. I think it
has been going well. Anybody who is interested can ask Andrew Overholt or I
for details.

Andrew

On Fri, Apr 8, 2016 at 10:52 AM, Douglas Turner  wrote:

> Emma,
>
> Thanks for doing this.
>
> I'm not sure whether something like this would work for platform
> engineering, but we'll keep an eye how things develop with Firefox and
> might consider it once we have some experience there.  I encourage you to
> report back here when Firefox starts using this system and has some lessons
> learned.
>
> I also want to thank all of the people that participated in this
> conversation.
>
> Doug
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Kartikaya Gupta]

On Fri, Apr 8, 2016 at 10:54 AM, Benjamin Smedberg
 wrote:
> On Thu, Apr 7, 2016 at 2:50 AM, L. David Baron  wrote:
>> Why it's important
>> What makes this problem important or urgent to fix?
>>
>
> Yes! If this isn't clear, who owns this? Either the module owner/peer, or a
> product manager, usually. Are there cases where we don't know?
>

In my experience this piece is often the most unclear. If we trying to
be more product-focused, then I think ultimately the product manager
should have the final say on the urgency of any user-visible bug.
However, in order to make an informed decision, they need to have
information about how frequently and under what circumstances the bug
occurs. Usually that information is not available without doing some
sort of time-consuming investigation as to the root cause. This
approach requires a high amount of bandwidth from the product
management team (they have to look at a lot of bugs) as well as the
dev team (lots of investigation into root causes).

In practice this tends decision tends to "gracefully degrade" into a
process where the module owner/peer looks at the bug, makes a guess as
to how bad it is, and requests tracking if it's worse than some
arbitrary/subjective threshold, and product management may not even
see it until much later (when it's already on aurora/beta, for
example). I'm not sure what we can do to improve this process exactly,
but I feel like there's room to make it more objective and clear-cut
without requiring extra bandwidth. I'm thinking of something along the
lines of a wiki page or some documentation that provides more
objective guidelines as to when a bug should be marked as tracking,
that the module owners/peers/triagers can reference when trying to
decide.

kats
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Chris Hofmann]

You might be able to create views into these triage queue's with
combinations of existing keywords and maybe a few added ones.  For example:

Can reproduce
Can others reproduce the problem described?

has the inverse represented by needs-testcase for example.  In the case of
the bug that needed nurturing people like alice white descend on piles of
bugs that need test cases, to then find regression ranges and and apply
techniques to build test cases.  We need to re-enforce that kind of process
and maybe give people like that another keyword like "has-testcase" so
engineers with domain knowledge can descend on that set of bugs.

Then

   Why it's important
What makes this problem important or urgent to fix?

combinations of keywords like "regression", "losing-users",
"sec-[critial/high] and other keywords plus some consistent priority
setting could offer good summations of different categories of important
bugs to stay on top of.

With constant triage its now possible to find good piles of bugs that have
test cases, are regressions, have been determined to be critical and are
"assigned to nob...@mozilla.org" then with these its possible to reset
organizational priorities and individual assignments.   All that happens
but in clumsy ways.  A good first step is to take dbaron's list of things
that need to be understood, see of we have existing markings that represent
or would help lead to understanding those aspects and formalize a bit of
process around that.   The regression keyword is one that get pretty good
use, but it still doesn't reflect the kinds of ways in which regressions
have been determined to be critical or are ok with just ignoring or pushing
out to some future date or never.   The platform triage is very focused on
trying to do just that.

-chofmann

On Thu, Apr 7, 2016 at 3:22 PM, Emma Humphries  wrote:

> First: there's that Bugzilla Anthropology project (
> https://wiki.mozilla.org/Bugzilla_Anthropology) thanks. I'd heard mention
> of it and asked around.
>
> What we found during the pilot, and the Platform Team has found in their
> triage, is that list of bugs with needinfo? is
>
> I'm worried that multiple queues in a component would be adding
> complexity, but this is something I'm willing experiment with.
>
> I don't think that picking a component or group of components we could try
> that style of triage on would block implementing the Triaged:Yes|No flag
> and UX improvements in Bugzilla.
>
> Would you be up for helping me run a short pilot of the multi-queue system
> you're describing?
>
> -- Emma
>
>
> On Wed, Apr 6, 2016 at 11:50 PM, L. David Baron  wrote:
>
>> On Wednesday 2016-04-06 18:47 -0700, Emma Humphries wrote:
>> > Following up on yesterday's email: I put together a draft second
>> proposal
>> > and shopped it around some, and now I want to bring that back into the
>> main
>> > discussion.
>> >
>> > The bullet point version of this is:
>> >
>> > * Add a binary field that components can use, TRIAGED (Y/N, T/F, +,-)
>> > * In the case of Firefox related components, have a consistent
>> definition
>> > of P1-P5 and make sure that triaged bugs have a Priority assigned
>>
>> > > On Tue, Mar 29, 2016 at 1:07 PM, Emma Humphries 
>> wrote:
>> > >> Today I’m asking for feedback on the plan which is posted at:
>> > >>
>> > >>
>> > >>
>> https://docs.google.com/document/d/1FFrtS0u6gNBE1mxsGJA9JLseJ_U6tW-1NJvHMq551ko
>>
>> I'm assuming the above URL represents the current proposal, although
>> given the "draft second proposal" wording above, I'm not sure if
>> that's the case.
>>
>>
>> I don't think the idea that a bug belongs in the triage queue if it
>> is untriaged and without a needinfo? is the right process.  I think,
>> instead, that there should be less emphasis on needinfo? to a
>> specific person, and more emphasis on grouping bugs into a small
>> number of categories of information that is needed, and allowing
>> people to triage those separate queues.
>>
>> Explaining why I think that requires a little bit of a digression:
>>
>> There are a number of separate things we want to understand in a bug
>> report, as I described in http://dbaron.org/log/20120816-bug-system :
>>
>> Understand description
>> Do the developers or triagers reading the bug understand
>> what the reporter is saying?
>>
>> Agree it's a bug
>> Do the module owners or peers or other authority agree that
>> the problem is a bug?
>>
>> Can reproduce
>> Can others reproduce the problem described?
>>
>> Why it's important
>> What makes this problem important or urgent to fix?
>>
>> How to fix
>> What should be done to fix the problem?
>>
>> (I'd much rather a bug report be editable text, with history
>> available, for answers to these or similar questions -- rather than
>> a stream of permanent comments.  But we seem stuck with the horrid
>> 

Re: Triage Plan for Firefox Components

[Emma Humphries]

First: there's that Bugzilla Anthropology project (
https://wiki.mozilla.org/Bugzilla_Anthropology) thanks. I'd heard mention
of it and asked around.

What we found during the pilot, and the Platform Team has found in their
triage, is that list of bugs with needinfo? is

I'm worried that multiple queues in a component would be adding complexity,
but this is something I'm willing experiment with.

I don't think that picking a component or group of components we could try
that style of triage on would block implementing the Triaged:Yes|No flag
and UX improvements in Bugzilla.

Would you be up for helping me run a short pilot of the multi-queue system
you're describing?

-- Emma


On Wed, Apr 6, 2016 at 11:50 PM, L. David Baron  wrote:

> On Wednesday 2016-04-06 18:47 -0700, Emma Humphries wrote:
> > Following up on yesterday's email: I put together a draft second proposal
> > and shopped it around some, and now I want to bring that back into the
> main
> > discussion.
> >
> > The bullet point version of this is:
> >
> > * Add a binary field that components can use, TRIAGED (Y/N, T/F, +,-)
> > * In the case of Firefox related components, have a consistent definition
> > of P1-P5 and make sure that triaged bugs have a Priority assigned
>
> > > On Tue, Mar 29, 2016 at 1:07 PM, Emma Humphries 
> wrote:
> > >> Today I’m asking for feedback on the plan which is posted at:
> > >>
> > >>
> > >>
> https://docs.google.com/document/d/1FFrtS0u6gNBE1mxsGJA9JLseJ_U6tW-1NJvHMq551ko
>
> I'm assuming the above URL represents the current proposal, although
> given the "draft second proposal" wording above, I'm not sure if
> that's the case.
>
>
> I don't think the idea that a bug belongs in the triage queue if it
> is untriaged and without a needinfo? is the right process.  I think,
> instead, that there should be less emphasis on needinfo? to a
> specific person, and more emphasis on grouping bugs into a small
> number of categories of information that is needed, and allowing
> people to triage those separate queues.
>
> Explaining why I think that requires a little bit of a digression:
>
> There are a number of separate things we want to understand in a bug
> report, as I described in http://dbaron.org/log/20120816-bug-system :
>
> Understand description
> Do the developers or triagers reading the bug understand
> what the reporter is saying?
>
> Agree it's a bug
> Do the module owners or peers or other authority agree that
> the problem is a bug?
>
> Can reproduce
> Can others reproduce the problem described?
>
> Why it's important
> What makes this problem important or urgent to fix?
>
> How to fix
> What should be done to fix the problem?
>
> (I'd much rather a bug report be editable text, with history
> available, for answers to these or similar questions -- rather than
> a stream of permanent comments.  But we seem stuck with the horrid
> stream-of-comments Bugzilla format, which means I try to ignore the
> comments as much as I can.  Then again, a 200 character summary is
> often good enough to answer the above 5 questions.  As with the rest
> of the Internet, don't read the comments.)
>
> Determining answers to any or all of the above might require
> multiple rounds of dialog, and some of them need to be understood in
> sequence rather than in parallel.  They also require very different
> sets of expertise to determine.  (Some of them require people with a
> bit of domain knowledge; some require detailed knowledge of relevant
> specifications.)  But they also don't require expertise from one
> person in particular, so needinfo? is not generally appropriate.  So
> the idea that bugs are in a queue unless they have a needinfo? seems
> wrong to me; I think a good triage process should involve queues
> representing the type of knowledge that's needed, rather than queues
> for a particular person to answer -- and people should be able to
> triage those queues (e.g., bugs that are untriaged because they lack
> information that allows others to reproduce, bugs that are untriaged
> because they require an expert in the relevant standard to decide
> whether our current behavior is correct or not -- state that could
> also be reflected in the bug) or the general queue of
> next-step-unknown bugs.
>
> While we can't get very far at all if we don't understand the
> description, it's often hard to make a decision about the priority
> until we understand how many users are affected (which sometimes
> requires the ability to reproduce), and about whether the bug
> described is actually a bug at all.  (Though there are also many
> cases where we can make decisions about the priority without some of
> these pieces of information.)  These are things that might be most
> efficiently determined by different people acting as part of the
> triage process, but the people aren't specific enough that needinfo?
> is the right 

Re: Triage Plan for Firefox Components

[Chris Hofmann]

re: Determining answers to any or all of the above might require
 multiple rounds of dialog, and some of them need to be understood in
 sequence rather than in parallel.  They also require very different
 sets of expertise to determine.

Yes!  We should set up systems and process and skills that encourage and
allow teams to "nuture" bugs, and get the answers to these questions.

Going into bug triage with the idea that primary and most important goal is
to get regression bugs "off the list", and to make (snap/quick) decisions
works against the process of finding important regressions, finding them
fast, and getting them fixed before we ship; which should be the main
reason we are doing all this work.

https://bugzilla.mozilla.org/show_bug.cgi?id=1249083 is an example.  seems
too bizarre to be an important or widespread problem, so ->works for me.
almost left for dead.  then resurrected and tied to an important problem
and fix.  bugs can often linger in these early stages for quite some time
before progress can be made, but we need to keep revisiting and keep trying
for breakthroughs in the information the dbaron lists.

Once the diagnosis gets better, the problems become more clear, then the
fixes can progress; often with great speed.

Streamline bugzilla to do these things and constantly remind all
participants that this is what we are trying to do.

-chofmann



> There are a number of separate things we want to understand in a bug
> report, as I described in http://dbaron.org/log/20120816-bug-system :
>
> Understand description
> Do the developers or triagers reading the bug understand
> what the reporter is saying?
>
> Agree it's a bug
> Do the module owners or peers or other authority agree that
> the problem is a bug?
>
> Can reproduce
> Can others reproduce the problem described?
>
> Why it's important
> What makes this problem important or urgent to fix?
>
> How to fix
> What should be done to fix the problem?
>
>
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[L. David Baron]

On Wednesday 2016-04-06 18:47 -0700, Emma Humphries wrote:
> Following up on yesterday's email: I put together a draft second proposal
> and shopped it around some, and now I want to bring that back into the main
> discussion.
> 
> The bullet point version of this is:
> 
> * Add a binary field that components can use, TRIAGED (Y/N, T/F, +,-)
> * In the case of Firefox related components, have a consistent definition
> of P1-P5 and make sure that triaged bugs have a Priority assigned

> > On Tue, Mar 29, 2016 at 1:07 PM, Emma Humphries  wrote:
> >> Today I’m asking for feedback on the plan which is posted at:
> >>
> >>
> >> https://docs.google.com/document/d/1FFrtS0u6gNBE1mxsGJA9JLseJ_U6tW-1NJvHMq551ko

I'm assuming the above URL represents the current proposal, although
given the "draft second proposal" wording above, I'm not sure if
that's the case.


I don't think the idea that a bug belongs in the triage queue if it
is untriaged and without a needinfo? is the right process.  I think,
instead, that there should be less emphasis on needinfo? to a
specific person, and more emphasis on grouping bugs into a small
number of categories of information that is needed, and allowing
people to triage those separate queues.

Explaining why I think that requires a little bit of a digression:

There are a number of separate things we want to understand in a bug
report, as I described in http://dbaron.org/log/20120816-bug-system :

Understand description
Do the developers or triagers reading the bug understand
what the reporter is saying?

Agree it's a bug
Do the module owners or peers or other authority agree that
the problem is a bug?

Can reproduce
Can others reproduce the problem described?

Why it's important
What makes this problem important or urgent to fix?

How to fix
What should be done to fix the problem? 

(I'd much rather a bug report be editable text, with history
available, for answers to these or similar questions -- rather than
a stream of permanent comments.  But we seem stuck with the horrid
stream-of-comments Bugzilla format, which means I try to ignore the
comments as much as I can.  Then again, a 200 character summary is
often good enough to answer the above 5 questions.  As with the rest
of the Internet, don't read the comments.)

Determining answers to any or all of the above might require
multiple rounds of dialog, and some of them need to be understood in
sequence rather than in parallel.  They also require very different
sets of expertise to determine.  (Some of them require people with a
bit of domain knowledge; some require detailed knowledge of relevant
specifications.)  But they also don't require expertise from one
person in particular, so needinfo? is not generally appropriate.  So
the idea that bugs are in a queue unless they have a needinfo? seems
wrong to me; I think a good triage process should involve queues
representing the type of knowledge that's needed, rather than queues
for a particular person to answer -- and people should be able to
triage those queues (e.g., bugs that are untriaged because they lack
information that allows others to reproduce, bugs that are untriaged
because they require an expert in the relevant standard to decide
whether our current behavior is correct or not -- state that could
also be reflected in the bug) or the general queue of
next-step-unknown bugs.

While we can't get very far at all if we don't understand the
description, it's often hard to make a decision about the priority
until we understand how many users are affected (which sometimes
requires the ability to reproduce), and about whether the bug
described is actually a bug at all.  (Though there are also many
cases where we can make decisions about the priority without some of
these pieces of information.)  These are things that might be most
efficiently determined by different people acting as part of the
triage process, but the people aren't specific enough that needinfo?
is the right tool.

-David

-- 
턞   L. David Baron http://dbaron.org/   턂
턢   Mozilla  https://www.mozilla.org/   턂
 Before I built a wall I'd ask to know
 What I was walling in or walling out,
 And to whom I was like to give offense.
   - Robert Frost, Mending Wall (1914)


signature.asc
Description: Digital signature
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Eric Rescorla]

Sorry to be dense, but if I understand correctly, you'd like to:

1. Have a policy that all of Gecko needs to triage bugs in a certain way.
2. Redefine how everyone defines priorities?

And you think that 24 hours is enough time to get consensus on that.

Do I have that right?

-Ekr


On Wed, Apr 6, 2016 at 10:47 PM, Emma Humphries  wrote:

> Following up on yesterday's email: I put together a draft second proposal
> and shopped it around some, and now I want to bring that back into the main
> discussion.
>
> The bullet point version of this is:
>
> * Add a binary field that components can use, TRIAGED (Y/N, T/F, +,-)
> * In the case of Firefox related components, have a consistent definition
> of P1-P5 and make sure that triaged bugs have a Priority assigned
>
> This has a couple of implications:
>
> This means I have to have a plan in place for dealing with Priority going
> from ad-hoc to one set of meanings for bugs in Firefox components.
>
> If any of you have worked with longitudinal social sciences data sets,
> this is a common thing. Values for fields change over time, and researcher
> consult documentation so that code consuming the data would work with the
> discontinuities. Some of this can be handled through bugzilla UI, so that
> components using the TRIAGED flag would display the description
> corresponding to P1-P5 and other components would not.
>
> We also need to go through all the existing whiteboard, keywords, and
> custom flags we are using for e10s and other projects that are being used
> to indicate importance.
>
> I'd like to finish up feedback on by the end of the working day Thursday
> the 6th (PST.)
>
Then we'll get to work on a solid specification for the work so we can
> start implementation sometime in Q2.
>
> Thanks.
>
> -- Emma
>
> On Tue, Apr 5, 2016 at 5:27 PM, Emma Humphries  wrote:
>
>> It's been a week since I asked for your comments on the plan for triage,
>> thank you.
>>
>> I'm going reply to some general comments on the plan, and outline next
>> steps.
>>
>> Ekt and others said that up to now, individual teams have owned how they
>> triage and prioritized bugs. Mozilla has made commitments to how we are
>> going to follow up with people filing bugs. Thus we need consistent
>> decisions across all the components that go into Firefox about bugs that we
>> can share back to non-Mozillans on bugs they file, so that we can get them
>> to contribute more high-quality bugs, and participate in other efforts in
>> support of the project and the Open Web. I'm aware I'm asking teams with
>> existing process to make a change, but it's for a global gain.
>>
>> Several people pointed out all the fields in Bugzilla that have and could
>> be used to manage priorities, such as priority and rank. But we don't use
>> the priority field consistently across the project. I've asked for teams to
>> document how they use Priority,
>> https://wiki.mozilla.org/Bugmasters/Projects/Folk_Knowledge/Priority_Field,
>> and you'll see how that varies.
>>
>> When I checked how the Priority field was used in Firefox-related
>> components, that distribution was:
>>
>> --- 460,362
>> P1   14,304
>> P2   15,971
>> P3   37,933
>> P44,204
>> P52,913
>>
>> The bulk of bugs in Firefox-related components are P3, most likely
>> because we have a bug filing form that defaults to P3 and that needs to be
>> fixed if it's still in use.
>>
>> Having to make what seemed like snap-decisions on bugs was also a point
>> of concern, but that's something the proposal had a work around for, using
>> needinfo? to defer a triage decision on a bug until enough questions were
>> answered. And since we made a commitment to make decisions on bugs, we need
>> back pressure on untriaged bugs.
>>
>> But from what I read, y'all are amenable to standardizing the priority
>> flag's use in Triage. Doing that would create a discontinuity in historical
>> data, but that's not an insurmountable problem, and we can document that
>> breakage for researchers using historical data.
>>
>> So next step is a second proposal, simplified, using Priority to
>> represent triage decisions.
>>
>> In addition, I'll want to remove several fields which are not useful, or
>> superfluous from the bug entry wizards. Priority is a field that should be
>> set by people triaging bugs, not entering them. We have a keyword
>> vocabulary which is more expressive than severity. And our bug entry forms
>> don't show the version affected, or the STR (steps to reproduce) flags
>> which means it's an extra edit to get the information relman needs into a
>> bug.
>>
>> Thank you again for your time and consideration as we make Bugzilla and
>> Firefox better for everyone.
>>
>> -- Emma Humphries
>>
>>
>> On Tue, Mar 29, 2016 at 1:07 PM, Emma Humphries  wrote:
>>
>>> tl;dr
>>>
>>> In Quarter Two I'm implementing the work we’ve been doing to improve
>>> triage, make actionable decisions on new bugs, and 

Re: Triage Plan for Firefox Components

[Emma Humphries]

On Fri, Apr 1, 2016 at 9:45 AM, James Graham  wrote:

> But once we have picked something, can we at least try to remove any UI
> that is more-or-less vestigial given that decision and, at least briefly,
> fight entropy by making things simpler and more consistent, rather than the
> reverse.



​Yes. I do want to do this as I work on the implementation of the changes
for triage.

-- Emma​
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Eric Rescorla]

On Tue, Apr 5, 2016 at 9:27 PM, Emma Humphries  wrote:

> It's been a week since I asked for your comments on the plan for triage,
> thank you.
>
> I'm going reply to some general comments on the plan, and outline next
> steps.
>
> Ekt and others said that up to now, individual teams have owned how they
> triage and prioritized bugs. Mozilla has made commitments to how we are
> going to follow up with people filing bugs.
>

Where were those commitments made? Can you send a link?



The bulk of bugs in Firefox-related components are P3, most likely because
> we have a bug filing form that defaults to P3 and that needs to be fixed if
> it's still in use.
>
> Having to make what seemed like snap-decisions on bugs was also a point of
> concern, but that's something the proposal had a work around for, using
> needinfo? to defer a triage decision on a bug until enough questions were
> answered. And since we made a commitment to make decisions on bugs, we need
> back pressure on untriaged bugs.
>
> But from what I read, y'all are amenable to standardizing the priority
> flag's use in Triage
>

I don't think this is a question of which *flag* to use so much as whether
it's useful to produce a new flat taxonomy which is redundant with the
existing priority mechanisms that teams are using, which in many cases are
richer than a three-level (now, soon, no-plan) hierarchy as you propose.

I think the fundamental problem here is that you're trying to design
something that might be useful for defects but isn't useful for a large
fraction of bugs which are actually a method of documenting planned new
work. Bug Bugzilla needs to work for all of these.

-Ekr


In addition, I'll want to remove several fields which are not useful, or
> superfluous from the bug entry wizards. Priority is a field that should be
> set by people triaging bugs, not entering them. We have a keyword
> vocabulary which is more expressive than severity. And our bug entry forms
> don't show the version affected, or the STR (steps to reproduce) flags
> which means it's an extra edit to get the information relman needs into a
> bug.
>
> Thank you again for your time and consideration as we make Bugzilla and
> Firefox better for everyone.
>
> -- Emma Humphries
>
>
> On Tue, Mar 29, 2016 at 1:07 PM, Emma Humphries  wrote:
>
>> tl;dr
>>
>> In Quarter Two I'm implementing the work we’ve been doing to improve
>> triage, make actionable decisions on new bugs, and prevent us from shipping
>> regressions in Firefox.
>>
>> Today I’m asking for feedback on the plan which is posted at:
>>
>>
>> https://docs.google.com/document/d/1FFrtS0u6gNBE1mxsGJA9JLseJ_U6tW-1NJvHMq551ko
>>
>> Allowing bugs to sit around without a decision on what we will do about
>> them sends the wrong message to Mozillans about how we treat bugs, how we
>> value their involvement, and reduces quality.
>>
>> The Firefox quality team (myself, Mike Hoye, Ryan VanderMeulen, Mark
>> Cote, and Benjamin Smedberg) want to make better assertions about the
>> quality of our releases by giving you tools to make clear decisions about
>> which bugs must be fixed for each release (urgent) and actively tracking
>> those bugs.
>> What We Learned From The Pilot Program
>>
>> During the past 6 weeks, we have prototyped and tested a triage process
>> with the DOM, Hello, and Developer Tools teams.
>>
>> Andrew Overholt, who participated in the pilot for the DOM team, said, “A
>> consistent bug triage process can help us spread the load of watching
>> incoming bugs and help avoid issues falling through the cracks."
>>
>> During the pilot, the DOM team uncovered critical bugs quickly so that
>> people could be assigned to them.
>>
>> The pilot groups also found that the triage process needs to be fast and
>> have tooling to make going through bugs fast. It’s easy to fall behind on
>> triage for a component, but if you stay up to date it will take no more
>> than 15 minutes a day.
>>
>> You can find the bugs we triaged during the pilot by looking for
>> whiteboard tags containing ‘btpp-’.
>>
>> It is also important to have consistent, shared definitions for
>> regression across components so triagers do not waste effort on mis-labeled
>> bugs.
>> Comments?
>>
>> I am posting this plan now for comment over the next week. I intend to
>> finalize the triage plan for implementation by Tuesday, April 5th. Feedback
>> and questions are welcome on the document, privately via email or IRC
>> (where I’m emceeaich) or on the bugmast...@mozilla.org mailing list.
>> Timeline
>>
>> January: finish finding component responsible parties
>>
>> February: pilot review of NEW bugs with four groups of components, draft
>> new process
>>
>> Now: comment period for new process, finalize process
>>
>> Q2: implement new process across all components involved in shipping
>> Firefox
>> Q3: all newly triaged bugs following the new process
>>
>> -- Emma Humphries, Bugmaster
>>
>
>
> 

Re: Triage Plan for Firefox Components

[Nicholas Alexander]

Emma,

On Tue, Apr 5, 2016 at 5:27 PM, Emma Humphries  wrote:

> It's been a week since I asked for your comments on the plan for triage,
> thank you.
>
> I'm going reply to some general comments on the plan, and outline next
> steps.
>

I just want to say that I have been following this thread, and that I
haven't been a fan of the thrust of the initiative, but I really appreciate
this summation and your effort to be transparent.


Nick
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Emma Humphries]

It's been a week since I asked for your comments on the plan for triage,
thank you.

I'm going reply to some general comments on the plan, and outline next
steps.

Ekt and others said that up to now, individual teams have owned how they
triage and prioritized bugs. Mozilla has made commitments to how we are
going to follow up with people filing bugs. Thus we need consistent
decisions across all the components that go into Firefox about bugs that we
can share back to non-Mozillans on bugs they file, so that we can get them
to contribute more high-quality bugs, and participate in other efforts in
support of the project and the Open Web. I'm aware I'm asking teams with
existing process to make a change, but it's for a global gain.

Several people pointed out all the fields in Bugzilla that have and could
be used to manage priorities, such as priority and rank. But we don't use
the priority field consistently across the project. I've asked for teams to
document how they use Priority,
https://wiki.mozilla.org/Bugmasters/Projects/Folk_Knowledge/Priority_Field,
and you'll see how that varies.

When I checked how the Priority field was used in Firefox-related
components, that distribution was:

--- 460,362
P1   14,304
P2   15,971
P3   37,933
P44,204
P52,913

The bulk of bugs in Firefox-related components are P3, most likely because
we have a bug filing form that defaults to P3 and that needs to be fixed if
it's still in use.

Having to make what seemed like snap-decisions on bugs was also a point of
concern, but that's something the proposal had a work around for, using
needinfo? to defer a triage decision on a bug until enough questions were
answered. And since we made a commitment to make decisions on bugs, we need
back pressure on untriaged bugs.

But from what I read, y'all are amenable to standardizing the priority
flag's use in Triage. Doing that would create a discontinuity in historical
data, but that's not an insurmountable problem, and we can document that
breakage for researchers using historical data.

So next step is a second proposal, simplified, using Priority to represent
triage decisions.

In addition, I'll want to remove several fields which are not useful, or
superfluous from the bug entry wizards. Priority is a field that should be
set by people triaging bugs, not entering them. We have a keyword
vocabulary which is more expressive than severity. And our bug entry forms
don't show the version affected, or the STR (steps to reproduce) flags
which means it's an extra edit to get the information relman needs into a
bug.

Thank you again for your time and consideration as we make Bugzilla and
Firefox better for everyone.

-- Emma Humphries


On Tue, Mar 29, 2016 at 1:07 PM, Emma Humphries  wrote:

> tl;dr
>
> In Quarter Two I'm implementing the work we’ve been doing to improve
> triage, make actionable decisions on new bugs, and prevent us from shipping
> regressions in Firefox.
>
> Today I’m asking for feedback on the plan which is posted at:
>
>
> https://docs.google.com/document/d/1FFrtS0u6gNBE1mxsGJA9JLseJ_U6tW-1NJvHMq551ko
>
> Allowing bugs to sit around without a decision on what we will do about
> them sends the wrong message to Mozillans about how we treat bugs, how we
> value their involvement, and reduces quality.
>
> The Firefox quality team (myself, Mike Hoye, Ryan VanderMeulen, Mark Cote,
> and Benjamin Smedberg) want to make better assertions about the quality of
> our releases by giving you tools to make clear decisions about which bugs
> must be fixed for each release (urgent) and actively tracking those bugs.
> What We Learned From The Pilot Program
>
> During the past 6 weeks, we have prototyped and tested a triage process
> with the DOM, Hello, and Developer Tools teams.
>
> Andrew Overholt, who participated in the pilot for the DOM team, said, “A
> consistent bug triage process can help us spread the load of watching
> incoming bugs and help avoid issues falling through the cracks."
>
> During the pilot, the DOM team uncovered critical bugs quickly so that
> people could be assigned to them.
>
> The pilot groups also found that the triage process needs to be fast and
> have tooling to make going through bugs fast. It’s easy to fall behind on
> triage for a component, but if you stay up to date it will take no more
> than 15 minutes a day.
>
> You can find the bugs we triaged during the pilot by looking for
> whiteboard tags containing ‘btpp-’.
>
> It is also important to have consistent, shared definitions for regression
> across components so triagers do not waste effort on mis-labeled bugs.
> Comments?
>
> I am posting this plan now for comment over the next week. I intend to
> finalize the triage plan for implementation by Tuesday, April 5th. Feedback
> and questions are welcome on the document, privately via email or IRC
> (where I’m emceeaich) or on the bugmast...@mozilla.org mailing list.
> Timeline
>
> January: finish finding component 

Re: Triage Plan for Firefox Components

[Justin Dolske]

A few comments, although I think others have touched on some of this
already.

I think my biggest concern is that this is conflating triage and
prioritization, which complicates things and introduces ambiguity. For
example, what would it even mean to have a "triage: fix-now" bug that's
also a P5? I'd much rather leave prioritization to the priority field. I'm
similarly dubious about the value of "not-urgent" and "wishlist". In part
because those also imply priorities, but also because this phrasing seems
like just kicking the can down the road on "making a decision".

It would seem simpler to just have a triage +/-/? flag to indicate if a
decision on triage has been made. Although we're still kind of tap-dancing
around what to do with the inevitable pile of "we just won't be able to fix
this soon bugs" -- I'm not sure a subtle triage flag effectively
communicates intent to people that don't live inside Bugzilla like we do.
It wouldn't be any worse than the status quo (and so is fine to trial), but
it should really just be a new status/resolution... The existing "WONTFIX"
and "INVALID" often convey the wrong meaning, a more accurate (and
gentler!) "DEFERRED" or "INACTIVE" or $BIKESHED_HERE is something I'd like
to see for this common condition.

I don't really understand the 3-month auto-timeout of bugs back to an
untriaged state. This seems like it's just creating work and makes Bugzilla
annoying. Nor is a one-size-fits all solution right here -- large /
long-term and small / short-term project areas will have different needs.
Grooming the list of triaged, prioritized bugs is important for teams to
do, and we should encourage them to do so, but starting off with this being
automatic seems premature.

I'm also concerned about bolting on yet more UI, which makes Bugzilla even
_more_ obtuse and complicated. But given all the feedback on the current
proposal, I suppose it's not worth commenting on here because I presume
this would all change anyway. This also seems like it should just be a
separate project.

Justin


On Tue, Mar 29, 2016 at 1:07 PM, Emma Humphries  wrote:

> tl;dr
>
> In Quarter Two I'm implementing the work we’ve been doing to improve
> triage, make actionable decisions on new bugs, and prevent us from shipping
> regressions in Firefox.
>
> Today I’m asking for feedback on the plan which is posted at:
>
>
> https://docs.google.com/document/d/1FFrtS0u6gNBE1mxsGJA9JLseJ_U6tW-1NJvHMq551ko
>
> Allowing bugs to sit around without a decision on what we will do about
> them sends the wrong message to Mozillans about how we treat bugs, how we
> value their involvement, and reduces quality.
>
> The Firefox quality team (myself, Mike Hoye, Ryan VanderMeulen, Mark Cote,
> and Benjamin Smedberg) want to make better assertions about the quality of
> our releases by giving you tools to make clear decisions about which bugs
> must be fixed for each release (urgent) and actively tracking those bugs.
> What We Learned From The Pilot Program
>
> During the past 6 weeks, we have prototyped and tested a triage process
> with the DOM, Hello, and Developer Tools teams.
>
> Andrew Overholt, who participated in the pilot for the DOM team, said, “A
> consistent bug triage process can help us spread the load of watching
> incoming bugs and help avoid issues falling through the cracks."
>
> During the pilot, the DOM team uncovered critical bugs quickly so that
> people could be assigned to them.
>
> The pilot groups also found that the triage process needs to be fast and
> have tooling to make going through bugs fast. It’s easy to fall behind on
> triage for a component, but if you stay up to date it will take no more
> than 15 minutes a day.
>
> You can find the bugs we triaged during the pilot by looking for
> whiteboard tags containing ‘btpp-’.
>
> It is also important to have consistent, shared definitions for regression
> across components so triagers do not waste effort on mis-labeled bugs.
> Comments?
>
> I am posting this plan now for comment over the next week. I intend to
> finalize the triage plan for implementation by Tuesday, April 5th. Feedback
> and questions are welcome on the document, privately via email or IRC
> (where I’m emceeaich) or on the bugmast...@mozilla.org mailing list.
> Timeline
>
> January: finish finding component responsible parties
>
> February: pilot review of NEW bugs with four groups of components, draft
> new process
>
> Now: comment period for new process, finalize process
>
> Q2: implement new process across all components involved in shipping
> Firefox
> Q3: all newly triaged bugs following the new process
>
> -- Emma Humphries, Bugmaster
>
> ___
> firefox-dev mailing list
> firefox-...@mozilla.org
> https://mail.mozilla.org/listinfo/firefox-dev
>
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Gervase Markham]

On 01/04/16 15:51, Mike Hommey wrote:
> Bug status is currently, IMHO, completely misused and thus useless:
> - people with editbug capability file as NEW by default. Why should a bug
>   I file in a component I'm not working on (because I noticed a bug
>   in Firefox) be NEW?
> - there is a long tail of bugs assigned to people that are not being worked
>   on (I should know, I have a lot of those, shame on me)
> 
> So it feels to me triage should replace/subsume it in some way.

I suspect they want to add a new field because changing bug statuses
seems like a massive change. Which it would be. However, not doing it
will leave us with two workflow widgets in Bugzilla instead of one, with
all the ambiguity that brings. In the long term, I see pain here.

If Bugzilla supported per-product workflow that might help. Is it worth
investing the BMO-hacking resources for this plan into that?

Gerv

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Chris Peterson]

On 4/1/16 10:57 AM, Emma Humphries wrote:

Could you add how your team maps the the priority flag, or link to a page
describing it?

https://wiki.mozilla.org/index.php?title=Bugmasters/Projects/Folk_Knowledge/Priority_Field=edit=1


This is interesting information! Do you have other "folk knowledge" from 
teams using Bugzilla? It may be useful to interview a number of 
engineers and managers about their teams' current Bugzilla workflows. 
And perhaps an etherpad for a big brain dump and bikeshedding. :)


If we can "pave some cowpaths" by standardizing proven patterns using 
our existing tools, there may be less resistance or bikeshedding to new 
proposals.


For example, here are some existing fields that different teams use 
(inconsistently) to prioritize future work:


- Priority
- Severity
- Rank (1–60)
- Votes
- Target Milestone
- Iteration
- custom tracking flags like tracking-e10s, firefox‑backlog, blocking-fx
- custom 'project flags' for B2G
- custom whiteboard tags
- various 'wanted' and priority keywords
- meta bugs
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Mike Hommey]

On Thu, Mar 31, 2016 at 06:47:32PM -0700, Emma Humphries wrote:
> What to call these has been a long thread in the document comments, but I'm
> starting to lean towards:
> 
> FIX-FOR-RELEASE
> FIX-SOON
> and
> BACKLOG
> 
> as well as adding (see the proposed edit) a FEATURE resolution for bugs
> which are feature tracking and individual features. I'd consider adding a
> consistency rule requiring a User Story field for bugs in that status.

If we're talking about a bmo-wide change, then a tri-state where one of
the state is bound to something being released, then that only leaves
two states for components that are not really attached to things making
it for a particular release, or are not released at all.

If we still get to use Pn and other related fields on top of that, why
are we adding another field? Not that I'm against adding fields, but as
someone else mentioned in the thread, we've done plenty of adding fields
in the past 18 years, and that's what kind of leads to the mess we are
in today. Adding a new field should be accompanied with removing
other(s).

On the other hand, I find it interesting that in some way, it overlaps
with the existing bug status (UNCONFIRMED, NEW, ASSIGNED), and I think
the document is dismissing it too quickly:

"because a bug can go from UNCONFIRMED to NEW, without us knowing how
important it is."

to which I want to answer with another question:

"then why not simply add status(es)?"

Bug status is currently, IMHO, completely misused and thus useless:
- people with editbug capability file as NEW by default. Why should a bug
  I file in a component I'm not working on (because I noticed a bug
  in Firefox) be NEW?
- there is a long tail of bugs assigned to people that are not being worked
  on (I should know, I have a lot of those, shame on me)

So it feels to me triage should replace/subsume it in some way.

Mike
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Steve Fink]

On 04/01/2016 08:32 AM, Gijs Kruitbosch wrote:



On 01/04/2016 16:16, Andrew McCreight wrote:

On Thu, Mar 31, 2016 at 4:01 PM, Chris Peterson 
wrote:

Anthony's Media Playback team has been using a simple and effective 
triage

system without special tags. All open bugs in the Audio/Video Playback
component are in one of four states at all times:

* Priority P1 bugs should be fixed ASAP.
* Priority P2 bugs are real bugs or features we want to fix soon.
* Priority P5 bugs are "patches accepted" bugs.
* Bugs without a priority are untriaged.



The drawback of indicating priorities in this way is that it is totally
opaque to anybody who is not on that particular subteam, let alone 
somebody
who is using Bugzilla for the first time to report breakage on a site 
they

use. If this was marked "backlog" or whatever instead of "P5", and there
was a link to an explanation of what "backlog" meant, then it would 
make it
much easier for people to figure out what is going on in any bug. I 
think

that is a big advantage of the proposed triaging system, even though I
think it is reasonable for anybody to be skeptical of adding even more
bells and whistles to the Bugzilla interface.

Andrew


Concur. Slight (related) aside: I've seen plenty of new bugreporters 
set priority to P5 and severity to critical - clearly, the highest 
number is the highest priority, right? In other words, I don't know 
that P1 being highest-prio is necessarily obvious to everyone.


Would an option be to just change the labels on the priority levels in 
the UI?


  P1 (fix for release)
  P2 (fix soon)
  P5 (backlog)

I am not for or against doing such a thing, I just wanted to raise the 
possibility.


If we do start consistently using the priority field for workflow, then 
I'd suggest removing it from the bug filing page for non-editbugs users 
(or some other determination of whether any value filled in is likely to 
be useful). It looks like it's already only in the Advanced Fields view, 
and I guess I can't easily tell whether it shows up for non-editbugs 
users, so perhaps this is already the case? (Just hiding it under 
Advanced isn't good enough IMO.)


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Emma Humphries]

While we're on the topic of how teams use the Priority Flag, could I ask
y'all do to something for me?

Could you add how your team maps the the priority flag, or link to a page
describing it?

https://wiki.mozilla.org/index.php?title=Bugmasters/Projects/Folk_Knowledge/Priority_Field=edit=1

Thank you!

On Fri, Apr 1, 2016 at 10:46 AM, Mats Palmgren  wrote:

> On 04/01/2016 10:35, Marco Bonardo wrote:
>
>> Fwiw, we (Awesome Search Team) also use Priority for the backlog, and we
>> consider P1, P2 and P5 the same as Media Playback team. So there is some
>> convergence about the meaning of those.
>> Though we also use P3 for "we care about this and will fix it, provided
>> there's no major priority bug to fix first".
>>
>
> This is roughly how we use P1-P5 in Layout as well:
> http://dbaron.org/log/20090120-bug-priorities
>
> /Mats
>
>
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Mats Palmgren]

On 04/01/2016 10:35, Marco Bonardo wrote:

Fwiw, we (Awesome Search Team) also use Priority for the backlog, and we
consider P1, P2 and P5 the same as Media Playback team. So there is some
convergence about the meaning of those.
Though we also use P3 for "we care about this and will fix it, provided
there's no major priority bug to fix first".


This is roughly how we use P1-P5 in Layout as well:
http://dbaron.org/log/20090120-bug-priorities

/Mats

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[James Graham]

On 01/04/16 01:02, Emma Humphries wrote:

I've responded to a similar comment in the google doc, but I'll repeat it
here.

Priority sounds like a great choice, but given that everyone's using the
Priority field their own way, there's enough heterogeneity in how it's used
to make it difficult.


I kind of feel like this is the story of Bugzilla. Every time there's a 
desire to do something slightly new the path of least resistance is to 
add yet another UI element for that one special case. Now 18 years later 
the interface is a confusing mess, and fields are either irrelevant or 
critically important depending on which component the bug happens to be in.


Which is not to argue for a particular solution here. I don't have a 
strong opinion. But once we have picked something, can we at least try 
to remove any UI that is more-or-less vestigial given that decision and, 
at least briefly, fight entropy by making things simpler and more 
consistent, rather than the reverse.

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Mike Hoye]

On 2016-04-01 11:32 AM, Gijs Kruitbosch wrote:



On 01/04/2016 16:16, Andrew McCreight wrote:


The drawback of indicating priorities in this way is that it is totally
opaque to anybody who is not on that particular subteam, let alone 
somebody
who is using Bugzilla for the first time to report breakage on a site 
they

use. If this was marked "backlog" or whatever instead of "P5", and there
was a link to an explanation of what "backlog" meant, then it would 
make it
much easier for people to figure out what is going on in any bug. I 
think

that is a big advantage of the proposed triaging system, even though I
think it is reasonable for anybody to be skeptical of adding even more
bells and whistles to the Bugzilla interface.



Concur. Slight (related) aside: I've seen plenty of new bugreporters 
set priority to P5 and severity to critical - clearly, the highest 
number is the highest priority, right? In other words, I don't know 
that P1 being highest-prio is necessarily obvious to everyone.


That, I suspect, is just a taste of the hidden costs of having different 
"bugzilla-dialects" across the entire organization.


Programmatically making a non-trivial assertion about the state of a 
single bug is really difficult now, so automating or dashboarding 
anything is always a per-team one-off throwaway. It's difficult to get 
aggregated information to managers, adds a bunch of friction between 
collaborating teams and increases the risk of human error.


Not to mention, it's a real barrier to participation. I'm invested in 
this because I want to make it easier for our community members to get 
involved productively.  I'm convinced that the same standard 
nomenclature that will make it easier for new, non-English-native 
participants to work in Bugzilla effectively will also make it easier 
for everyone at a manager level or higher to communicate and coordinate 
efforts.


- mhoye


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Andrew McCreight]

On Fri, Apr 1, 2016 at 8:33 AM, Milan Sreckovic 
wrote:

> Valid point.
>
> Being able to tell the status of “other team’s bugs” is critical for some
> number of people that look at bugs across all teams.  A number of
> directors, release management, I’m sure a sizeable number.  I would still
> guess a minor though; I know I am completely unaffected by the P’s and Q’s
> that the media team uses, and don’t care if they are different from what
> E10S is using.  And whatever they have seems to be working for them.
>
> If it is important for everything to be the same, we need to consider the
> multiplier.  For example, if it’s going to make life twice as easy for
> Lawrence (hi Lawrence), but make it 10% more difficult for all the
> developers - I’d say we shouldn’t do it.  If it’s going to let us have
> better decision making because we can extract some data out of bugzilla
> that we otherwise couldn’t, then we probably should do it (“it” = be
> consistent.)
>
> Sometimes we want to do things because it makes things clean and pretty.
> When that’s the only reason to do something, we should be cautious (it
> would be clean and pretty if everybody worked 9-5, and out of an office,
> right?).  If clean and pretty leads to better data, and that data is
> required and used for better decision making, different story.
>

It is also important for everybody who files a bug in a component handled
by a team they are not on. We obviously want people who are not on the,
say, graphics team to file graphics bugs. We should have some way to
indicate to these bug reporters what sort of priority their bug is being
treated as, in a way that they can understand.

Teams can spend a little bit of time learning to select "backlog" or
whatever instead of "P5", or they can spend time in every bug communicating
to reporters what "P5" means to their particular team. The former is more
efficient over all. Or we can have the status quo, where bugs that aren't
being actively worked on have a status that is just confusing to bug
reporters, which discourages people from filing more bugs.

Andrew



>
> —
> - Milan
>
>
>
> > On Apr 1, 2016, at 11:16 , Andrew McCreight 
> wrote:
> >
> > On Thu, Mar 31, 2016 at 4:01 PM, Chris Peterson 
> > wrote:
> >
> >> Anthony's Media Playback team has been using a simple and effective
> triage
> >> system without special tags. All open bugs in the Audio/Video Playback
> >> component are in one of four states at all times:
> >>
> >> * Priority P1 bugs should be fixed ASAP.
> >> * Priority P2 bugs are real bugs or features we want to fix soon.
> >> * Priority P5 bugs are "patches accepted" bugs.
> >> * Bugs without a priority are untriaged.
> >>
> >
> > The drawback of indicating priorities in this way is that it is totally
> > opaque to anybody who is not on that particular subteam, let alone
> somebody
> > who is using Bugzilla for the first time to report breakage on a site
> they
> > use. If this was marked "backlog" or whatever instead of "P5", and there
> > was a link to an explanation of what "backlog" meant, then it would make
> it
> > much easier for people to figure out what is going on in any bug. I think
> > that is a big advantage of the proposed triaging system, even though I
> > think it is reasonable for anybody to be skeptical of adding even more
> > bells and whistles to the Bugzilla interface.
> >
> > Andrew
> >
> >
> >> P3 and P4 are not used. :) P5 sends a pretty clear message to people
> that
> >> we will not fix that issue any time soon. However, the P5 list is pretty
> >> clean because we aggressively WONTFIX bugs we truly don't want to fix
> >> instead of letting them linger.
> >>
> >> Bugzilla already has a lot of fields. It seems like new workflows could
> be
> >> built with a streamlined frontend UI without changing Bugzilla's
> database
> >> schema. The advantage of codifying existing workflows instead of adding
> new
> >> fields is that existing bugs will be compatible with the new system.
> >>
> >> IIRC, the Fennec team also tracked the "Someone is working on this"
> >> proposed in Emma's plan by assigning owners to all bugs, but developers
> >> would change their bug's status from NEW to ASSIGNED only when they
> began
> >> actively working on the bug.
> >>
> >> ___
> >> dev-platform mailing list
> >> dev-platform@lists.mozilla.org
> >> https://lists.mozilla.org/listinfo/dev-platform
> >>
> > ___
> > dev-platform mailing list
> > dev-platform@lists.mozilla.org
> > https://lists.mozilla.org/listinfo/dev-platform
>
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Milan Sreckovic]

Valid point.

Being able to tell the status of “other team’s bugs” is critical for some 
number of people that look at bugs across all teams.  A number of directors, 
release management, I’m sure a sizeable number.  I would still guess a minor 
though; I know I am completely unaffected by the P’s and Q’s that the media 
team uses, and don’t care if they are different from what E10S is using.  And 
whatever they have seems to be working for them.

If it is important for everything to be the same, we need to consider the 
multiplier.  For example, if it’s going to make life twice as easy for Lawrence 
(hi Lawrence), but make it 10% more difficult for all the developers - I’d say 
we shouldn’t do it.  If it’s going to let us have better decision making 
because we can extract some data out of bugzilla that we otherwise couldn’t, 
then we probably should do it (“it” = be consistent.)

Sometimes we want to do things because it makes things clean and pretty.  When 
that’s the only reason to do something, we should be cautious (it would be 
clean and pretty if everybody worked 9-5, and out of an office, right?).  If 
clean and pretty leads to better data, and that data is required and used for 
better decision making, different story.

—
- Milan



> On Apr 1, 2016, at 11:16 , Andrew McCreight  wrote:
> 
> On Thu, Mar 31, 2016 at 4:01 PM, Chris Peterson 
> wrote:
> 
>> Anthony's Media Playback team has been using a simple and effective triage
>> system without special tags. All open bugs in the Audio/Video Playback
>> component are in one of four states at all times:
>> 
>> * Priority P1 bugs should be fixed ASAP.
>> * Priority P2 bugs are real bugs or features we want to fix soon.
>> * Priority P5 bugs are "patches accepted" bugs.
>> * Bugs without a priority are untriaged.
>> 
> 
> The drawback of indicating priorities in this way is that it is totally
> opaque to anybody who is not on that particular subteam, let alone somebody
> who is using Bugzilla for the first time to report breakage on a site they
> use. If this was marked "backlog" or whatever instead of "P5", and there
> was a link to an explanation of what "backlog" meant, then it would make it
> much easier for people to figure out what is going on in any bug. I think
> that is a big advantage of the proposed triaging system, even though I
> think it is reasonable for anybody to be skeptical of adding even more
> bells and whistles to the Bugzilla interface.
> 
> Andrew
> 
> 
>> P3 and P4 are not used. :) P5 sends a pretty clear message to people that
>> we will not fix that issue any time soon. However, the P5 list is pretty
>> clean because we aggressively WONTFIX bugs we truly don't want to fix
>> instead of letting them linger.
>> 
>> Bugzilla already has a lot of fields. It seems like new workflows could be
>> built with a streamlined frontend UI without changing Bugzilla's database
>> schema. The advantage of codifying existing workflows instead of adding new
>> fields is that existing bugs will be compatible with the new system.
>> 
>> IIRC, the Fennec team also tracked the "Someone is working on this"
>> proposed in Emma's plan by assigning owners to all bugs, but developers
>> would change their bug's status from NEW to ASSIGNED only when they began
>> actively working on the bug.
>> 
>> ___
>> dev-platform mailing list
>> dev-platform@lists.mozilla.org
>> https://lists.mozilla.org/listinfo/dev-platform
>> 
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Andrew McCreight]

On Thu, Mar 31, 2016 at 4:01 PM, Chris Peterson 
wrote:

> Anthony's Media Playback team has been using a simple and effective triage
> system without special tags. All open bugs in the Audio/Video Playback
> component are in one of four states at all times:
>
> * Priority P1 bugs should be fixed ASAP.
> * Priority P2 bugs are real bugs or features we want to fix soon.
> * Priority P5 bugs are "patches accepted" bugs.
> * Bugs without a priority are untriaged.
>

The drawback of indicating priorities in this way is that it is totally
opaque to anybody who is not on that particular subteam, let alone somebody
who is using Bugzilla for the first time to report breakage on a site they
use. If this was marked "backlog" or whatever instead of "P5", and there
was a link to an explanation of what "backlog" meant, then it would make it
much easier for people to figure out what is going on in any bug. I think
that is a big advantage of the proposed triaging system, even though I
think it is reasonable for anybody to be skeptical of adding even more
bells and whistles to the Bugzilla interface.

Andrew


> P3 and P4 are not used. :) P5 sends a pretty clear message to people that
> we will not fix that issue any time soon. However, the P5 list is pretty
> clean because we aggressively WONTFIX bugs we truly don't want to fix
> instead of letting them linger.
>
> Bugzilla already has a lot of fields. It seems like new workflows could be
> built with a streamlined frontend UI without changing Bugzilla's database
> schema. The advantage of codifying existing workflows instead of adding new
> fields is that existing bugs will be compatible with the new system.
>
> IIRC, the Fennec team also tracked the "Someone is working on this"
> proposed in Emma's plan by assigning owners to all bugs, but developers
> would change their bug's status from NEW to ASSIGNED only when they began
> actively working on the bug.
>
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Milan Sreckovic]

> On Mar 31, 2016, at 18:22 , Daniel Veditz  wrote:
> 
> On Thu, Mar 31, 2016 at 12:28 PM, Milan Sreckovic  > wrote:
> I’m going to start and keep arguing that we do not want to have an explicit 
> name for that largest bucket of “wishlist” bugs, and should instead have it 
> marked by the absence of a tag.
> 
> ​What distinguishes a bug that has not been triaged from a bug that has been 
> triaged and (mentally?) put into the third bucket if there's no explicit 
> marker?​

I understood there was an “untriaged” status set by default.

On the other hand, the latest choice of “backlog” is non-confrontational enough 
that, for the purposes of the naming discussion, is close enough to “blank" for 
me not to argue.

—
- Milan

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Kartikaya Gupta]

It seems to me that when this bug program was started, it had these
two goals (quoted from Emma's previous email):

"First, we want
to make better assertions about the quality of our releases by making clear
decisions about which bugs must be fixed for each release (urgent) and
actively tracking those bugs. The other is the number of good bugs filed by
our community. Filing a bug report is a gateway to greater participation in
the Mozilla project, and we owe it to our community to make quick and clear
decisions about each bug."

Somewhere along the way this got conflated with "all teams must use
the same process to indicate how bugs got triaged". While I'm sure
that doing so provides additional benefits, I'm not sure it's
necessary to accomplish the goals stated above, and might actually be
counterproductive initially. Some teams already have processes in
place for triaging and annotating bugs, and have been using their
process for quite some time. Forcing them into a new process seems
unnecessary if what they are doing already works for them. I think
what's more important is that each component have an owner (or
owners), and that the triage process for that component is documented
somewhere and followed. For those teams that don't already have a
triage process or that are looking to change their process, I think
it's a great idea to have a canned "this is the recommended approach
we know has worked for some teams" ready for them to use. Dictating
specific process rather than allowing teams to find their own way to
meet the requirements is overkill.

If you do want to expand the scope of this proposal to add more
uniformity across Mozilla then that should be explicitly stated as a
goal. Personally, I think that might be better done as a follow-up
effort after looking at the variety of triage processes that different
teams use and finding something that is general enough to work for
everybody.

Cheers,
kats

On Fri, Apr 1, 2016 at 4:35 AM, Marco Bonardo  wrote:
> On Fri, Apr 1, 2016 at 2:02 AM, Emma Humphries  wrote:
>
>> Priority sounds like a great choice, but given that everyone's using the
>> Priority field their own way, there's enough heterogeneity in how it's used
>> to make it difficult.
>>
>
> Fwiw, we (Awesome Search Team) also use Priority for the backlog, and we
> consider P1, P2 and P5 the same as Media Playback team. So there is some
> convergence about the meaning of those.
> Though we also use P3 for "we care about this and will fix it, provided
> there's no major priority bug to fix first". This helps planning future
> work, by retriaging P3s into P2s when the list of P2s shrinks or new goals
> are made. The P5 pool would be too large to pick things out of it.
> Merging P4 into P5 wouldn't make much of a difference. Both are things we
> recognize are valid bugs, but it's very unlikely we'll ever have time to
> fix them.
>
> Imo, merging everything below P2 into a single backlog is a mistake,
> there's quite a huge difference between "we want to fix this once there's
> time but it's not critical now" to "we are unlikely to ever have time to
> fix this". I personally feel the need for a "P3" category.
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Marco Bonardo]

On Fri, Apr 1, 2016 at 2:02 AM, Emma Humphries  wrote:

> Priority sounds like a great choice, but given that everyone's using the
> Priority field their own way, there's enough heterogeneity in how it's used
> to make it difficult.
>

Fwiw, we (Awesome Search Team) also use Priority for the backlog, and we
consider P1, P2 and P5 the same as Media Playback team. So there is some
convergence about the meaning of those.
Though we also use P3 for "we care about this and will fix it, provided
there's no major priority bug to fix first". This helps planning future
work, by retriaging P3s into P2s when the list of P2s shrinks or new goals
are made. The P5 pool would be too large to pick things out of it.
Merging P4 into P5 wouldn't make much of a difference. Both are things we
recognize are valid bugs, but it's very unlikely we'll ever have time to
fix them.

Imo, merging everything below P2 into a single backlog is a mistake,
there's quite a huge difference between "we want to fix this once there's
time but it's not critical now" to "we are unlikely to ever have time to
fix this". I personally feel the need for a "P3" category.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Emma Humphries]

What to call these has been a long thread in the document comments, but I'm
starting to lean towards:

FIX-FOR-RELEASE
FIX-SOON
and
BACKLOG

as well as adding (see the proposed edit) a FEATURE resolution for bugs
which are feature tracking and individual features. I'd consider adding a
consistency rule requiring a User Story field for bugs in that status.


On Thu, Mar 31, 2016 at 12:28 PM, Milan Sreckovic 
wrote:

> This may be somewhat long winded.  I will make it in the context of Gecko
> graphics, because that’s where I have the most data and experience.  It may
> or may not apply to other components.
>
> Reviewing all the incoming bugs, in a timely matter, is very important to
> us.  Over the past few years, the graphics team fixed about half the bugs
> that came in (roughly, we fix a thousand out of the two thousand that come
> in.)  The main goal of any kind of a bug triage process is thus to choose
> the right half of the bugs to fix, and spend as little time as possible on
> the rest.
>
> With that in mind we started a much less documented and much more minimal
> process in 2015.  I don’t know that we have the data to back the “avoid
> issues falling between the cracks”, but it certainly seems that way.  One
> data point we do have - the average amount of time it took to fix the bugs
> triaged in 2015 was almost half of that for 2014 and the previous four
> years.
>
> This to illustrate that I believe in the bug triage, looking at bugs as
> they come in, and making some quick decisions how to proceed.
>
> I’m also a big believer in lightweight processes and minimal
> documentation, so there are a few comments I have on what the document
> below describes, and in general.
>
> The more states we have, the more not-so-useful conversation we’ll have
> about assigning those states.  I’m glad to see that we have a small number,
> currently named fix-now, non-urgent and wishlist (the naming is in flux and
> being argued in the document.)  I’m mentally mapping these to “blocking the
> release”, “can’t ship too many releases with this” and “I hope we
> eventually fix this”, but perhaps there is a different interpretation.
>
> I would expect to see the majority of the graphics bugs end up in the last
> group.  Why?  Since you never plan for the full capacity, as that actually
> reduces your throughput, and since we only fix half the bugs that come in,
> it stands to reason that we should not have even close to half of the bugs
> in the first two categories.  In other words, we want to be fixing some of
> the “wishlist” bugs.  And we need to be very careful about not making the
> fix-now turn into “we really should fix this”, and only allow the “team
> works on nothing else while there are fix-now bugs open”.  Otherwise, well,
> it loses its value.
>
> Step aside - my thoughts on the “How Triage Will Work in Bugzilla”
> section.  I would stick with the definition of the states and have
> dashboards that show the bug counts in different categories for different
> teams.  The current description is a bit too detailed and inflexible.  It
> suggests that we can figure out the best way to do this before we’ve even
> started doing it (the pilot program non-withstanding), and it mixes the
> mechanics with the goals.
>
> I’m going to start and keep arguing that we do not want to have an
> explicit name for that largest bucket of “wishlist” bugs, and should
> instead have it marked by the absence of a tag.  This is not about the
> heavily involved community, those that will stick around regardless of what
> we do to them, and anybody that reads this e-mail.  This is about people
> that are reporting their first bugs, that are occasionally involved, who
> are vital to our success.  If I was one of those, and I started seeing that
> most, if not all, of my bugs are marked as wishlist or deferred or
> in-copious-spare-time, I’m going to get discouraged and stop doing it.
> After all, I don’t seem to be valuably contributing, because I’m just
> telling you things you don’t care about.  Or, I could start arguing in the
> bug, that this should be higher priority, and fill up the comments with
> non-technical information.
>
> Getting close to a full page, I’ll stop now.  I’m available for live
> conversations on the topic :)
>
> —
> - Milan
>
>
>
> > On Mar 29, 2016, at 16:07 , Emma Humphries  wrote:
> >
> > tl;dr
> >
> > In Quarter Two I'm implementing the work we’ve been doing to improve
> > triage, make actionable decisions on new bugs, and prevent us from
> shipping
> > regressions in Firefox.
> >
> > Today I’m asking for feedback on the plan which is posted at:
> >
> >
> https://docs.google.com/document/d/1FFrtS0u6gNBE1mxsGJA9JLseJ_U6tW-1NJvHMq551ko
> >
> > Allowing bugs to sit around without a decision on what we will do about
> > them sends the wrong message to Mozillans about how we treat bugs, how we
> > value their involvement, and reduces quality.
> >
> > The Firefox 

Re: Triage Plan for Firefox Components

[Emma Humphries]

On Thu, Mar 31, 2016 at 12:28 PM, Milan Sreckovic 
wrote:

> Or, I could start arguing in the bug, that this should be higher priority,
> and fill up the comments with non-technical information.



​This reminds me, we have filtering tools to mute abusive and unproductive
comments in bugs.

https://wiki.mozilla.org/BMO/comment_tagging

-- Emma

​
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Emma Humphries]

I've responded to a similar comment in the google doc, but I'll repeat it
here.

Priority sounds like a great choice, but given that everyone's using the
Priority field their own way, there's enough heterogeneity in how it's used
to make it difficult.

If I was to take that approach, I'm concerned about what it would do with
historical data. We'd also have the P3, and P4 values hanging round like
vestigial limbs.

FYI, in FFx related components.

--- 460,362
P1   14,304
P2   15,971
P3   37,933
P44,204
P52,913

The bulk of bugs in FFx related components are P3.

-- Emma

On Thu, Mar 31, 2016 at 4:01 PM, Chris Peterson 
wrote:

> On 3/31/16 3:22 PM, Daniel Veditz wrote:
>
>> ​We get that now, with no marker at all. The only real difference I see
>> with a marker is that people will catch on sooner whereas now it takes a
>> while until they realize they are being ignored. They eventually get
>> discouraged​ or upset either way. Might even be better with a marker (for
>> some people) because at least they have some acknowledgement that someone
>> has looked at the bug even if they disagree about the importance. We may
>> have misunderstood, and thus mis-triaged, the bug and an explicit marker
>> might trigger the attempt to clarify sooner.
>>
>
> Anthony's Media Playback team has been using a simple and effective triage
> system without special tags. All open bugs in the Audio/Video Playback
> component are in one of four states at all times:
>
> * Priority P1 bugs should be fixed ASAP.
> * Priority P2 bugs are real bugs or features we want to fix soon.
> * Priority P5 bugs are "patches accepted" bugs.
> * Bugs without a priority are untriaged.
>
> P3 and P4 are not used. :) P5 sends a pretty clear message to people that
> we will not fix that issue any time soon. However, the P5 list is pretty
> clean because we aggressively WONTFIX bugs we truly don't want to fix
> instead of letting them linger.
>
> Bugzilla already has a lot of fields. It seems like new workflows could be
> built with a streamlined frontend UI without changing Bugzilla's database
> schema. The advantage of codifying existing workflows instead of adding new
> fields is that existing bugs will be compatible with the new system.
>
> IIRC, the Fennec team also tracked the "Someone is working on this"
> proposed in Emma's plan by assigning owners to all bugs, but developers
> would change their bug's status from NEW to ASSIGNED only when they began
> actively working on the bug.
>
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Chris Peterson]

On 3/31/16 3:22 PM, Daniel Veditz wrote:

​We get that now, with no marker at all. The only real difference I see
with a marker is that people will catch on sooner whereas now it takes a
while until they realize they are being ignored. They eventually get
discouraged​ or upset either way. Might even be better with a marker (for
some people) because at least they have some acknowledgement that someone
has looked at the bug even if they disagree about the importance. We may
have misunderstood, and thus mis-triaged, the bug and an explicit marker
might trigger the attempt to clarify sooner.


Anthony's Media Playback team has been using a simple and effective 
triage system without special tags. All open bugs in the Audio/Video 
Playback component are in one of four states at all times:


* Priority P1 bugs should be fixed ASAP.
* Priority P2 bugs are real bugs or features we want to fix soon.
* Priority P5 bugs are "patches accepted" bugs.
* Bugs without a priority are untriaged.

P3 and P4 are not used. :) P5 sends a pretty clear message to people 
that we will not fix that issue any time soon. However, the P5 list is 
pretty clean because we aggressively WONTFIX bugs we truly don't want to 
fix instead of letting them linger.


Bugzilla already has a lot of fields. It seems like new workflows could 
be built with a streamlined frontend UI without changing Bugzilla's 
database schema. The advantage of codifying existing workflows instead 
of adding new fields is that existing bugs will be compatible with the 
new system.


IIRC, the Fennec team also tracked the "Someone is working on this" 
proposed in Emma's plan by assigning owners to all bugs, but developers 
would change their bug's status from NEW to ASSIGNED only when they 
began actively working on the bug.

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Daniel Veditz]

On Thu, Mar 31, 2016 at 12:28 PM, Milan Sreckovic 
wrote:

> I’m going to start and keep arguing that we do not want to have an
> explicit name for that largest bucket of “wishlist” bugs, and should
> instead have it marked by the absence of a tag.


​What distinguishes a bug that has not been triaged from a bug that has
been triaged and (mentally?) put into the third bucket if there's no
explicit marker?​

This is about people that are reporting their first bugs, that are
> occasionally involved, who are vital to our success.  If I was one of
> those, and I started seeing that most, if not all, of my bugs are marked as
> wishlist or deferred or in-copious-spare-time, I’m going to get discouraged
> and stop doing it.  After all, I don’t seem to be valuably contributing,
> because I’m just telling you things you don’t care about.  Or, I could
> start arguing in the bug, that this should be higher priority, and fill up
> the comments with non-technical information.
>

​We get that now, with no marker at all. The only real difference I see
with a marker is that people will catch on sooner whereas now it takes a
while until they realize they are being ignored. They eventually get
discouraged​ or upset either way. Might even be better with a marker (for
some people) because at least they have some acknowledgement that someone
has looked at the bug even if they disagree about the importance. We may
have misunderstood, and thus mis-triaged, the bug and an explicit marker
might trigger the attempt to clarify sooner.

The people who are going to demand attention for their super important
critical blocker bug are likely to do that either way. A triage marker
might help lance the boil sooner rather than let frustration build up to
explosive levels.

-
​Dan Veditz​
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Milan Sreckovic]

This may be somewhat long winded.  I will make it in the context of Gecko 
graphics, because that’s where I have the most data and experience.  It may or 
may not apply to other components.

Reviewing all the incoming bugs, in a timely matter, is very important to us.  
Over the past few years, the graphics team fixed about half the bugs that came 
in (roughly, we fix a thousand out of the two thousand that come in.)  The main 
goal of any kind of a bug triage process is thus to choose the right half of 
the bugs to fix, and spend as little time as possible on the rest.

With that in mind we started a much less documented and much more minimal 
process in 2015.  I don’t know that we have the data to back the “avoid issues 
falling between the cracks”, but it certainly seems that way.  One data point 
we do have - the average amount of time it took to fix the bugs triaged in 2015 
was almost half of that for 2014 and the previous four years.

This to illustrate that I believe in the bug triage, looking at bugs as they 
come in, and making some quick decisions how to proceed.

I’m also a big believer in lightweight processes and minimal documentation, so 
there are a few comments I have on what the document below describes, and in 
general.

The more states we have, the more not-so-useful conversation we’ll have about 
assigning those states.  I’m glad to see that we have a small number, currently 
named fix-now, non-urgent and wishlist (the naming is in flux and being argued 
in the document.)  I’m mentally mapping these to “blocking the release”, “can’t 
ship too many releases with this” and “I hope we eventually fix this”, but 
perhaps there is a different interpretation.

I would expect to see the majority of the graphics bugs end up in the last 
group.  Why?  Since you never plan for the full capacity, as that actually 
reduces your throughput, and since we only fix half the bugs that come in, it 
stands to reason that we should not have even close to half of the bugs in the 
first two categories.  In other words, we want to be fixing some of the 
“wishlist” bugs.  And we need to be very careful about not making the fix-now 
turn into “we really should fix this”, and only allow the “team works on 
nothing else while there are fix-now bugs open”.  Otherwise, well, it loses its 
value.

Step aside - my thoughts on the “How Triage Will Work in Bugzilla” section.  I 
would stick with the definition of the states and have dashboards that show the 
bug counts in different categories for different teams.  The current 
description is a bit too detailed and inflexible.  It suggests that we can 
figure out the best way to do this before we’ve even started doing it (the 
pilot program non-withstanding), and it mixes the mechanics with the goals.

I’m going to start and keep arguing that we do not want to have an explicit 
name for that largest bucket of “wishlist” bugs, and should instead have it 
marked by the absence of a tag.  This is not about the heavily involved 
community, those that will stick around regardless of what we do to them, and 
anybody that reads this e-mail.  This is about people that are reporting their 
first bugs, that are occasionally involved, who are vital to our success.  If I 
was one of those, and I started seeing that most, if not all, of my bugs are 
marked as wishlist or deferred or in-copious-spare-time, I’m going to get 
discouraged and stop doing it.  After all, I don’t seem to be valuably 
contributing, because I’m just telling you things you don’t care about.  Or, I 
could start arguing in the bug, that this should be higher priority, and fill 
up the comments with non-technical information.

Getting close to a full page, I’ll stop now.  I’m available for live 
conversations on the topic :)

—
- Milan



> On Mar 29, 2016, at 16:07 , Emma Humphries  wrote:
> 
> tl;dr
> 
> In Quarter Two I'm implementing the work we’ve been doing to improve
> triage, make actionable decisions on new bugs, and prevent us from shipping
> regressions in Firefox.
> 
> Today I’m asking for feedback on the plan which is posted at:
> 
> https://docs.google.com/document/d/1FFrtS0u6gNBE1mxsGJA9JLseJ_U6tW-1NJvHMq551ko
> 
> Allowing bugs to sit around without a decision on what we will do about
> them sends the wrong message to Mozillans about how we treat bugs, how we
> value their involvement, and reduces quality.
> 
> The Firefox quality team (myself, Mike Hoye, Ryan VanderMeulen, Mark Cote,
> and Benjamin Smedberg) want to make better assertions about the quality of
> our releases by giving you tools to make clear decisions about which bugs
> must be fixed for each release (urgent) and actively tracking those bugs.
> What We Learned From The Pilot Program
> 
> During the past 6 weeks, we have prototyped and tested a triage process
> with the DOM, Hello, and Developer Tools teams.
> 
> Andrew Overholt, who participated in the pilot for the DOM team, said, “A
> consistent bug triage 

Re: Triage Plan for Firefox Components

[Lawrence Mandel]

Softvision also makes use of the feature keyword as one way to identify new
feature work to test in upcoming releases.

Lawrence

On Thu, Mar 31, 2016 at 10:49 AM, Milan Sreckovic 
wrote:

> We do have a feature keyword today.  While it may be most used for the
> documentation purposes, the feedback graphics team got when we started
> using it to tag feature requests was positive.  As in, it’s OK to use for
> that.
> —
> - Milan
>
>
>
> > On Mar 29, 2016, at 22:32 , Emma Humphries  wrote:
> >
> > On Tue, Mar 29, 2016 at 5:09 PM, Eric Rescorla  wrote:
> >
> >> On a more substantive, less procedural note, this seems to be designed
> for
> >> a particular workflow in which there is an assumption that bugs are for
> >> immediate processing. However, in may cases we use bugs as placeholders
> or
> >> assemble big dependency trees of all the bugs that are needed to do a
> large
> >> complex feature. From this perspective, a three-tier system of "urgent",
> >> "non-urgent", and "wishlist" is either a regression from more
> fine-grained
> >> systems such as dependency trees/priorities or is redundant with them.
> This
> >> is especially true for long-running efforts. In other words, this may
> be a
> >> useful change for some components while not being useful for others. For
> >> the specific case of NSS (where I currently do a lot of my work) this
> >> doesn't seem like it would be a helpful change.
> >
> >
> > ​This is where it would be nice to have a way of saying, feature vs. bug
> as
> > part of the triage process, even it that's not a clean separation to
> make,
> > because that could get long running feature work out of the triage
> process,
> > but I don't have an immediate answer for this. It may be that we change
> the
> > scope of components this applies to.
> >
> > I'll follow up with Doug Turner to see if changing the scope of this for
> > platform related bugs is warranted.
> >
> > ​
> > --
> > ​ Emma​
> > ___
> > dev-platform mailing list
> > dev-platform@lists.mozilla.org
> > https://lists.mozilla.org/listinfo/dev-platform
>
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Axel Hecht]

Hi Emma,

for those of us that are addicted to data: You have about a 1000 bugs of 
data, and I'd love to hear some of the good parts, and maybe also some 
of the bad parts.


Also, you tested on three teams, and you report a success story from 
one. Could you frame that a bit? Is that within the expectations, or 
above or below?


Axel

On 29/03/16 22:07, Emma Humphries wrote:

tl;dr

In Quarter Two I'm implementing the work we’ve been doing to improve
triage, make actionable decisions on new bugs, and prevent us from shipping
regressions in Firefox.

Today I’m asking for feedback on the plan which is posted at:

https://docs.google.com/document/d/1FFrtS0u6gNBE1mxsGJA9JLseJ_U6tW-1NJvHMq551ko

Allowing bugs to sit around without a decision on what we will do about
them sends the wrong message to Mozillans about how we treat bugs, how we
value their involvement, and reduces quality.

The Firefox quality team (myself, Mike Hoye, Ryan VanderMeulen, Mark Cote,
and Benjamin Smedberg) want to make better assertions about the quality of
our releases by giving you tools to make clear decisions about which bugs
must be fixed for each release (urgent) and actively tracking those bugs.
What We Learned From The Pilot Program

During the past 6 weeks, we have prototyped and tested a triage process
with the DOM, Hello, and Developer Tools teams.

Andrew Overholt, who participated in the pilot for the DOM team, said, “A
consistent bug triage process can help us spread the load of watching
incoming bugs and help avoid issues falling through the cracks."

During the pilot, the DOM team uncovered critical bugs quickly so that
people could be assigned to them.

The pilot groups also found that the triage process needs to be fast and
have tooling to make going through bugs fast. It’s easy to fall behind on
triage for a component, but if you stay up to date it will take no more
than 15 minutes a day.

You can find the bugs we triaged during the pilot by looking for whiteboard
tags containing ‘btpp-’.

It is also important to have consistent, shared definitions for regression
across components so triagers do not waste effort on mis-labeled bugs.
Comments?

I am posting this plan now for comment over the next week. I intend to
finalize the triage plan for implementation by Tuesday, April 5th. Feedback
and questions are welcome on the document, privately via email or IRC
(where I’m emceeaich) or on the bugmast...@mozilla.org mailing list.
Timeline

January: finish finding component responsible parties

February: pilot review of NEW bugs with four groups of components, draft
new process

Now: comment period for new process, finalize process

Q2: implement new process across all components involved in shipping Firefox
Q3: all newly triaged bugs following the new process

-- Emma Humphries, Bugmaster



___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Mike Hoye]

On 2016-03-29 8:09 PM, Eric Rescorla wrote:


On a more substantive, less procedural note, this seems to be designed 
for a particular workflow in which there is an assumption that bugs 
are for immediate processing. However, in may cases we use bugs as 
placeholders or assemble big dependency trees of all the bugs that are 
needed to do a large complex feature.
We're definitely talking about untriaged incoming bugs here, not 
placeholder or planning bugs, though it's true that the doc doesn't call 
that out explicitly. I'll add a comment to that effect.


- mhoye
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Emma Humphries]

On Tue, Mar 29, 2016 at 5:09 PM, Eric Rescorla  wrote:

> On a more substantive, less procedural note, this seems to be designed for
> a particular workflow in which there is an assumption that bugs are for
> immediate processing. However, in may cases we use bugs as placeholders or
> assemble big dependency trees of all the bugs that are needed to do a large
> complex feature. From this perspective, a three-tier system of "urgent",
> "non-urgent", and "wishlist" is either a regression from more fine-grained
> systems such as dependency trees/priorities or is redundant with them. This
> is especially true for long-running efforts. In other words, this may be a
> useful change for some components while not being useful for others. For
> the specific case of NSS (where I currently do a lot of my work) this
> doesn't seem like it would be a helpful change.


​This is where it would be nice to have a way of saying, feature vs. bug as
part of the triage process, even it that's not a clean separation to make,
because that could get long running feature work out of the triage process,
but I don't have an immediate answer for this. It may be that we change the
scope of components this applies to.

I'll follow up with Doug Turner to see if changing the scope of this for
platform related bugs is warranted.

​
 --
​ Emma​
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Eric Rescorla]

Hmmm... Who do you believe is the decider here and what do you believe is
the decision procedure for these components? Generally, the way things work
isn't that Firefox promulgates policy for how other components handle their
bugs.

On a more substantive, less procedural note, this seems to be designed for
a particular workflow in which there is an assumption that bugs are for
immediate processing. However, in may cases we use bugs as placeholders or
assemble big dependency trees of all the bugs that are needed to do a large
complex feature. From this perspective, a three-tier system of "urgent",
"non-urgent", and "wishlist" is either a regression from more fine-grained
systems such as dependency trees/priorities or is redundant with them. This
is especially true for long-running efforts. In other words, this may be a
useful change for some components while not being useful for others. For
the specific case of NSS (where I currently do a lot of my work) this
doesn't seem like it would be a helpful change.

-Ekr



On Tue, Mar 29, 2016 at 4:33 PM, Emma Humphries  wrote:

> Components that feed into FFx, so that's Core, Toolbox, Firefox, Fennec
> and the pieces of platform we're using to ship.
>
> While there's not a 'Gecko' component in bugzilla, it does cover the
> components there which are Gecko.
>
> -- Emma
>
> On Tue, Mar 29, 2016 at 3:16 PM, Eric Rescorla  wrote:
>
>> I'm trying to figure out the scope of this proposal. Are you expecting it
>> to apply merely to Firefox or to Gecko as well?
>>
>> -Ekr
>>
>>
>> On Tue, Mar 29, 2016 at 1:07 PM, Emma Humphries  wrote:
>>
>>> tl;dr
>>>
>>> In Quarter Two I'm implementing the work we’ve been doing to improve
>>> triage, make actionable decisions on new bugs, and prevent us from shipping
>>> regressions in Firefox.
>>>
>>> Today I’m asking for feedback on the plan which is posted at:
>>>
>>>
>>> https://docs.google.com/document/d/1FFrtS0u6gNBE1mxsGJA9JLseJ_U6tW-1NJvHMq551ko
>>>
>>> Allowing bugs to sit around without a decision on what we will do about
>>> them sends the wrong message to Mozillans about how we treat bugs, how we
>>> value their involvement, and reduces quality.
>>>
>>> The Firefox quality team (myself, Mike Hoye, Ryan VanderMeulen, Mark
>>> Cote, and Benjamin Smedberg) want to make better assertions about the
>>> quality of our releases by giving you tools to make clear decisions about
>>> which bugs must be fixed for each release (urgent) and actively tracking
>>> those bugs.
>>> What We Learned From The Pilot Program
>>>
>>> During the past 6 weeks, we have prototyped and tested a triage process
>>> with the DOM, Hello, and Developer Tools teams.
>>>
>>> Andrew Overholt, who participated in the pilot for the DOM team, said,
>>> “A consistent bug triage process can help us spread the load of watching
>>> incoming bugs and help avoid issues falling through the cracks."
>>>
>>> During the pilot, the DOM team uncovered critical bugs quickly so that
>>> people could be assigned to them.
>>>
>>> The pilot groups also found that the triage process needs to be fast and
>>> have tooling to make going through bugs fast. It’s easy to fall behind on
>>> triage for a component, but if you stay up to date it will take no more
>>> than 15 minutes a day.
>>>
>>> You can find the bugs we triaged during the pilot by looking for
>>> whiteboard tags containing ‘btpp-’.
>>>
>>> It is also important to have consistent, shared definitions for
>>> regression across components so triagers do not waste effort on mis-labeled
>>> bugs.
>>> Comments?
>>>
>>> I am posting this plan now for comment over the next week. I intend to
>>> finalize the triage plan for implementation by Tuesday, April 5th. Feedback
>>> and questions are welcome on the document, privately via email or IRC
>>> (where I’m emceeaich) or on the bugmast...@mozilla.org mailing list.
>>> Timeline
>>>
>>> January: finish finding component responsible parties
>>>
>>> February: pilot review of NEW bugs with four groups of components, draft
>>> new process
>>>
>>> Now: comment period for new process, finalize process
>>>
>>> Q2: implement new process across all components involved in shipping
>>> Firefox
>>> Q3: all newly triaged bugs following the new process
>>>
>>> -- Emma Humphries, Bugmaster
>>>
>>> ___
>>> firefox-dev mailing list
>>> firefox-...@mozilla.org
>>> https://mail.mozilla.org/listinfo/firefox-dev
>>>
>>>
>>
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Emma Humphries]

Components that feed into FFx, so that's Core, Toolbox, Firefox, Fennec and
the pieces of platform we're using to ship.

While there's not a 'Gecko' component in bugzilla, it does cover the
components there which are Gecko.

-- Emma

On Tue, Mar 29, 2016 at 3:16 PM, Eric Rescorla  wrote:

> I'm trying to figure out the scope of this proposal. Are you expecting it
> to apply merely to Firefox or to Gecko as well?
>
> -Ekr
>
>
> On Tue, Mar 29, 2016 at 1:07 PM, Emma Humphries  wrote:
>
>> tl;dr
>>
>> In Quarter Two I'm implementing the work we’ve been doing to improve
>> triage, make actionable decisions on new bugs, and prevent us from shipping
>> regressions in Firefox.
>>
>> Today I’m asking for feedback on the plan which is posted at:
>>
>>
>> https://docs.google.com/document/d/1FFrtS0u6gNBE1mxsGJA9JLseJ_U6tW-1NJvHMq551ko
>>
>> Allowing bugs to sit around without a decision on what we will do about
>> them sends the wrong message to Mozillans about how we treat bugs, how we
>> value their involvement, and reduces quality.
>>
>> The Firefox quality team (myself, Mike Hoye, Ryan VanderMeulen, Mark
>> Cote, and Benjamin Smedberg) want to make better assertions about the
>> quality of our releases by giving you tools to make clear decisions about
>> which bugs must be fixed for each release (urgent) and actively tracking
>> those bugs.
>> What We Learned From The Pilot Program
>>
>> During the past 6 weeks, we have prototyped and tested a triage process
>> with the DOM, Hello, and Developer Tools teams.
>>
>> Andrew Overholt, who participated in the pilot for the DOM team, said, “A
>> consistent bug triage process can help us spread the load of watching
>> incoming bugs and help avoid issues falling through the cracks."
>>
>> During the pilot, the DOM team uncovered critical bugs quickly so that
>> people could be assigned to them.
>>
>> The pilot groups also found that the triage process needs to be fast and
>> have tooling to make going through bugs fast. It’s easy to fall behind on
>> triage for a component, but if you stay up to date it will take no more
>> than 15 minutes a day.
>>
>> You can find the bugs we triaged during the pilot by looking for
>> whiteboard tags containing ‘btpp-’.
>>
>> It is also important to have consistent, shared definitions for
>> regression across components so triagers do not waste effort on mis-labeled
>> bugs.
>> Comments?
>>
>> I am posting this plan now for comment over the next week. I intend to
>> finalize the triage plan for implementation by Tuesday, April 5th. Feedback
>> and questions are welcome on the document, privately via email or IRC
>> (where I’m emceeaich) or on the bugmast...@mozilla.org mailing list.
>> Timeline
>>
>> January: finish finding component responsible parties
>>
>> February: pilot review of NEW bugs with four groups of components, draft
>> new process
>>
>> Now: comment period for new process, finalize process
>>
>> Q2: implement new process across all components involved in shipping
>> Firefox
>> Q3: all newly triaged bugs following the new process
>>
>> -- Emma Humphries, Bugmaster
>>
>> ___
>> firefox-dev mailing list
>> firefox-...@mozilla.org
>> https://mail.mozilla.org/listinfo/firefox-dev
>>
>>
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Triage Plan for Firefox Components

[Eric Rescorla]

I'm trying to figure out the scope of this proposal. Are you expecting it
to apply merely to Firefox or to Gecko as well?

-Ekr


On Tue, Mar 29, 2016 at 1:07 PM, Emma Humphries  wrote:

> tl;dr
>
> In Quarter Two I'm implementing the work we’ve been doing to improve
> triage, make actionable decisions on new bugs, and prevent us from shipping
> regressions in Firefox.
>
> Today I’m asking for feedback on the plan which is posted at:
>
>
> https://docs.google.com/document/d/1FFrtS0u6gNBE1mxsGJA9JLseJ_U6tW-1NJvHMq551ko
>
> Allowing bugs to sit around without a decision on what we will do about
> them sends the wrong message to Mozillans about how we treat bugs, how we
> value their involvement, and reduces quality.
>
> The Firefox quality team (myself, Mike Hoye, Ryan VanderMeulen, Mark Cote,
> and Benjamin Smedberg) want to make better assertions about the quality of
> our releases by giving you tools to make clear decisions about which bugs
> must be fixed for each release (urgent) and actively tracking those bugs.
> What We Learned From The Pilot Program
>
> During the past 6 weeks, we have prototyped and tested a triage process
> with the DOM, Hello, and Developer Tools teams.
>
> Andrew Overholt, who participated in the pilot for the DOM team, said, “A
> consistent bug triage process can help us spread the load of watching
> incoming bugs and help avoid issues falling through the cracks."
>
> During the pilot, the DOM team uncovered critical bugs quickly so that
> people could be assigned to them.
>
> The pilot groups also found that the triage process needs to be fast and
> have tooling to make going through bugs fast. It’s easy to fall behind on
> triage for a component, but if you stay up to date it will take no more
> than 15 minutes a day.
>
> You can find the bugs we triaged during the pilot by looking for
> whiteboard tags containing ‘btpp-’.
>
> It is also important to have consistent, shared definitions for regression
> across components so triagers do not waste effort on mis-labeled bugs.
> Comments?
>
> I am posting this plan now for comment over the next week. I intend to
> finalize the triage plan for implementation by Tuesday, April 5th. Feedback
> and questions are welcome on the document, privately via email or IRC
> (where I’m emceeaich) or on the bugmast...@mozilla.org mailing list.
> Timeline
>
> January: finish finding component responsible parties
>
> February: pilot review of NEW bugs with four groups of components, draft
> new process
>
> Now: comment period for new process, finalize process
>
> Q2: implement new process across all components involved in shipping
> Firefox
> Q3: all newly triaged bugs following the new process
>
> -- Emma Humphries, Bugmaster
>
> ___
> firefox-dev mailing list
> firefox-...@mozilla.org
> https://mail.mozilla.org/listinfo/firefox-dev
>
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform