That SGTM On Fri, Oct 25, 2019 at 4:18 PM Robert Bradshaw <[email protected]> wrote:
> +1 to both. > > On Fri, Oct 25, 2019 at 3:58 PM Valentyn Tymofieiev <[email protected]> > wrote: > > > > On Fri, Oct 25, 2019 at 3:39 PM Kenneth Knowles <[email protected]> wrote: > >> > >> Suppose, hypothetically, we say that if Fix Version is set, then > P0/Blocker and P1/Critical block release and lower priorities get bumped. > > > > > > +1 to Kenn's suggestion. In addition, we can discourage setting Fix > version for non-critical issues before issues are fixed. > > > >> > >> > >> Most likely the release manager still pings and asks about all those > before bumping. Which means that in effect they were part of the burn down > and do block the release in the sense that they must be re-triaged away to > the next release. I would prefer less work for the release manager and more > emphasis on the default being nonblocking. > >> > >> One very different possibility is to ignore Fix Version on open bugs > and use a different search query as the burndown, auto bump everything that > didn't make it. > > > > This may create a situation where an issue will eventually be closed, > but Fix Version not updated, and confuse the users who will rely "Fix > Version" to find which release actually fixes the issue. A pass over open > bugs with a Fix Version set to next release (as currently done by a release > manager) helps to make sure that unfixed bugs won't have Fix Version tag of > the upcoming release. > > > >> > >> Kenn > >> > >> On Fri, Oct 25, 2019, 14:16 Robert Bradshaw <[email protected]> > wrote: > >>> > >>> I'm fine with that, but in that case we should have a priority for > >>> release blockers, below which bugs get automatically bumped to the > >>> next release (and which becomes the burndown list). > >>> > >>> On Fri, Oct 25, 2019 at 1:58 PM Kenneth Knowles <[email protected]> > wrote: > >>> > > >>> > My takeaway from this thread is that priorities should have a shared > community intuition and/or policy around how they are treated, which could > eventually be formalized into SLOs. > >>> > > >>> > At a practical level, I do think that build breaks are higher > priority than release blockers. If you are on this thread but not looking > at the PR, here is the verbiage I added about urgency: > >>> > > >>> > P0/Blocker: "A P0 issue is more urgent than simply blocking the next > release" > >>> > P1/Critical: "Most critical bugs should block release" > >>> > P2/Major: "No special urgency is associated" > >>> > ... > >>> > > >>> > Kenn > >>> > > >>> > On Fri, Oct 25, 2019 at 11:46 AM Robert Bradshaw < > [email protected]> wrote: > >>> >> > >>> >> We cut a release every 6 weeks, according to schedule, making it > easy > >>> >> to plan for, and the release manager typically sends out a warning > >>> >> email to remind everyone. I don't think it makes sense to do that > for > >>> >> every ticket. Blockers should be reserved for things we really > >>> >> shouldn't release without. > >>> >> > >>> >> On Fri, Oct 25, 2019 at 11:33 AM Pablo Estrada <[email protected]> > wrote: > >>> >> > > >>> >> > I mentioned on the PR that I had been using the 'blocker' > priority along with the 'fix version' field to mark issues that I want to > get in the release. > >>> >> > Of course, this little practice of mine only matters much around > release branch cutting time - and has been useful for me to track which > things I want to ensure getting into the release / bump to the next /etc. > >>> >> > I've also found it to be useful as a way to communicate with the > release manager without having to sync directly. > >>> >> > > >>> >> > What would be a reasonable way to tell the release manager "I'd > like to get this feature in. please talk to me if you're about to cut the > branch" - that also uses the priorities appropriately? - and that allows > the release manager to know when a fix version is "more optional" / "less > optional"? > >>> >> > > >>> >> > On Wed, Oct 23, 2019 at 12:20 PM Kenneth Knowles <[email protected]> > wrote: > >>> >> >> > >>> >> >> I finally got around to writing some of this up. It is minimal. > Feedback is welcome, especially if what I have written does not accurately > represent the community's approach. > >>> >> >> > >>> >> >> https://github.com/apache/beam/pull/9862 > >>> >> >> > >>> >> >> Kenn > >>> >> >> > >>> >> >> On Mon, Feb 11, 2019 at 3:21 PM Daniel Oliveira < > [email protected]> wrote: > >>> >> >>> > >>> >> >>> Ah, sorry, I missed that Alex was just quoting from our Jira > installation (didn't read his email closely enough). Also I wasn't aware > about those pages on our website. > >>> >> >>> > >>> >> >>> Seeing as we do have definitions for our priorities, I guess my > main request would be that they be made more discoverable somehow. I don't > think the tooltips are reliable, and the pages on the website are > informative, but hard to find. Since it feels a bit lazy to say "this isn't > discoverable enough" without suggesting any improvements, I'd like to > propose these two changes: > >>> >> >>> > >>> >> >>> 1. We should write a Beam Jira Guide with basic information > about our Jira. I think the bug priorities should go in here, but also > anything else we would want someone to know before filing any Jira issues, > like how our components are organized or what the different issue types > mean. This guide could either be written in the website or the wiki, but I > think it should definitely be linked in > https://beam.apache.org/contribute/ so that newcomers read it before > getting their Jira account approved. The goal here being to have a > reference for the basics of our Jira since at the moment it doesn't seem > like we have anything for this. > >>> >> >>> > >>> >> >>> 2. The existing info on Post-commit and pre-commit policies > doesn't seem very discoverable to someone monitoring the Pre/Post-commits. > I've reported a handful of test-failures already and haven't seen this link > mentioned much. We should try to find a way to funnel people towards this > link when there's an issue, the same way we try to funnel people towards > the contribution guide when they write a PR. As a note, while writing this > email I remembered this link that someone gave me before ( > https://s.apache.org/beam-test-failure). That mentions the Post-commit > policies page, so maybe it's just a matter of pasting that all over our > Jenkins builds whenever we have a failing test? > >>> >> >>> > >>> >> >>> PS: I'm also definitely for SLOs, but I figure it's probably > better discussed in a separate thread so I'm trying to stick to the subject > of priority definitions. > >>> >> >>> > >>> >> >>> On Mon, Feb 11, 2019 at 9:17 AM Scott Wegner <[email protected]> > wrote: > >>> >> >>>> > >>> >> >>>> Thanks for driving this discussion. I also was not aware of > these existing definitions. Once we agree on the terms, let's add them to > our Contributor Guide and start using them. > >>> >> >>>> > >>> >> >>>> +1 in general; I like both Alex and Kenn's definitions; > Additional wordsmithing could be moved to a Pull Request. Can we make the > definitions useful for both the person filing a bug, and the assignee, i.e. > >>> >> >>>> > >>> >> >>>> <Priority Level>: <Criteria for what types of issues should be > assigned>. <Expectations for responding to a Priority Level issue> > >>> >> >>>> > >>> >> >>>> On Sun, Feb 10, 2019 at 7:49 PM Kenneth Knowles < > [email protected]> wrote: > >>> >> >>>>> > >>> >> >>>>> The content that Alex posted* is the definition from our Jira > installation anyhow. > >>> >> >>>>> > >>> >> >>>>> I just searched around, and there's > https://community.atlassian.com/t5/Jira-questions/According-to-Jira-What-is-Blocker-Critical-Major-Minor-and/qaq-p/668774 > which makes clear that this is really user-defined, since Jira has many > deployments with their own configs. > >>> >> >>>>> > >>> >> >>>>> I guess what I want to know about this thread is what action > is being proposed? > >>> >> >>>>> > >>> >> >>>>> Previously, there was a thread that resulted in > https://beam.apache.org/contribute/precommit-policies/ and > https://beam.apache.org/contribute/postcommits-policies/. These have test > failures and flakes as Critical. I agree with Alex that these should be > Blocker. They disrupt the work of the entire community, so we need to drop > everything and get green again. > >>> >> >>>>> > >>> >> >>>>> Other than that, I think what you - Daniel - are suggesting > is that the definition might be best expressed as SLOs. I asked on > [email protected] about how we could have those and the answer is the > homebrew > https://svn.apache.org/repos/infra/infrastructure/trunk/projects/status/sla/jira/. > If anyone has time to dig into that and see if it can work for us, that > would be cool. > >>> >> >>>>> > >>> >> >>>>> Kenn > >>> >> >>>>> > >>> >> >>>>> *Blocker: Blocks development and/or testing work, production > could not run > >>> >> >>>>> Critical: Crashes, loss of data, severe memory leak. > >>> >> >>>>> Major (Default): Major loss of function. > >>> >> >>>>> Minor: Minor loss of function, or other problem where easy > workaround is present. > >>> >> >>>>> Trivial: Trivial Cosmetic problem like misspelt words or > misaligned text. > >>> >> >>>>> > >>> >> >>>>> > >>> >> >>>>> On Sun, Feb 10, 2019 at 7:20 PM Daniel Oliveira < > [email protected]> wrote: > >>> >> >>>>>> > >>> >> >>>>>> Are there existing meanings for the priorities in Jira > already? I wasn't able to find any info on either the Beam website or wiki > about it, so I've just been prioritizing issues based on gut feeling. If > not, I think having some well-defined priorities would be nice, at least > for our test-failures, and especially if we wanna have some SLOs like I've > seen being thrown about. > >>> >> >>>>>> > >>> >> >>>>>> On Fri, Feb 8, 2019 at 3:06 PM Kenneth Knowles < > [email protected]> wrote: > >>> >> >>>>>>> > >>> >> >>>>>>> I've been thinking about this since working on the release. > If I ignore the names I think: > >>> >> >>>>>>> > >>> >> >>>>>>> P0: get paged, stop whatever you planned on doing, work > late to fix > >>> >> >>>>>>> P1: continually update everyone on status and shouldn't sit > around unassigned > >>> >> >>>>>>> P2: most things here; they can be planned or picked up by > whomever > >>> >> >>>>>>> P3: nice-to-have things, maybe starter tasks or lesser > cleanup, but no driving need > >>> >> >>>>>>> Sometimes there's P4 but I don't value it. Often P3 is a > deprioritized thing from P2, so more involved and complex, while P4 is > something easy and not important filed just as a reminder. Either way, they > are both not on the main path of work. > >>> >> >>>>>>> > >>> >> >>>>>>> I looked into it and the Jira priority scheme determines > the set of priorities as well as the default. Ours is shared by 635 > projects. Probably worth keeping. The default priority is Major which would > correspond with P2. We can expect the default to be where most issues end > up. > >>> >> >>>>>>> > >>> >> >>>>>>> P0 == Blocker: get paged, stop whatever you planned on > doing, work late to fix > >>> >> >>>>>>> P1 == Critical: continually update everyone on status and > shouldn't sit around unassigned > >>> >> >>>>>>> P0 == Major (default): most things here; they can be > planned or picked up by whomever > >>> >> >>>>>>> P3 == Minor: nice-to-have things, maybe starter tasks or > lesser cleanup, but no driving need > >>> >> >>>>>>> Trivial: Maybe this is attractive to newcomers as it makes > it sound easy. > >>> >> >>>>>>> > >>> >> >>>>>>> Kenn > >>> >> >>>>>>> > >>> >> >>>>>>> On Thu, Feb 7, 2019 at 4:08 PM Alex Amato < > [email protected]> wrote: > >>> >> >>>>>>>> > >>> >> >>>>>>>> Hello Beam community, I was thinking about this and found > some information to share/discuss. Would it be possible to confirm my > thinking on this: > >>> >> >>>>>>>> > >>> >> >>>>>>>> There are 5 priorities in the JIRA system today (tooltip > link): > >>> >> >>>>>>>> > >>> >> >>>>>>>> Blocker Blocks development and/or testing work, production > could not run > >>> >> >>>>>>>> Critical Crashes, loss of data, severe memory leak. > >>> >> >>>>>>>> Major Major loss of function. > >>> >> >>>>>>>> Minor Minor loss of function, or other problem where easy > workaround is present. > >>> >> >>>>>>>> Trivial Cosmetic problem like misspelt words or misaligned > text. > >>> >> >>>>>>>> > >>> >> >>>>>>>> How should JIRA issues be prioritized for pre/post commit > test failures? > >>> >> >>>>>>>> > >>> >> >>>>>>>> I think Blocker > >>> >> >>>>>>>> > >>> >> >>>>>>>> What about the flakey failures? > >>> >> >>>>>>>> > >>> >> >>>>>>>> Blocker as well? > >>> >> >>>>>>>> > >>> >> >>>>>>>> How should non test issues be prioritized? (E.g. feature > to implement or bugs not regularly breaking tests). > >>> >> >>>>>>>> > >>> >> >>>>>>>> I suggest Minor, but its not clear how to distinguish > between these. > >>> >> >>>>>>>> > >>> >> >>>>>>>> Below is my thinking: But I wanted to know what the > Apache/Beam community generally thinks about these priorities. > >>> >> >>>>>>>> > >>> >> >>>>>>>> Blocker: Expect to be paged. Production systems are down. > >>> >> >>>>>>>> Critical: Expect to be contacted by email or a bot to fix > this. > >>> >> >>>>>>>> Major: Some loss of function in the repository, can issues > that need to be addressed soon are here. > >>> >> >>>>>>>> Minor: Most issues will be here, important issues within > this will get picked up and completed. FRs, bugs. > >>> >> >>>>>>>> Trivial: Unlikely to be implemented, far too many issues > in this category. FRs, bugs. > >>> >> >>>>>>>> > >>> >> >>>>>>>> Thanks for helping to clear this up > >>> >> >>>>>>>> Alex > >>> >> >>>> > >>> >> >>>> > >>> >> >>>> > >>> >> >>>> -- > >>> >> >>>> > >>> >> >>>> > >>> >> >>>> > >>> >> >>>> > >>> >> >>>> Got feedback? tinyurl.com/swegner-feedback >
