Can we get some PMC opinions on this? According to https://www.apache.org/dev/infra-contact#request-checklist, it is highly recommended to have your opinions when filing a request.
--Mikhail Have feedback <http://go/migryz-feedback>? On Wed, Jul 11, 2018 at 6:01 PM Anton Kedin <ke...@google.com> wrote: > I think this looks good, we should enable the plugin and try it out. > Concrete details of the follow-up tasks (auto-assignment, triage, and > dashboarding) will probably depend on how functional the plugin is and what > the test failures data looks like. > > Regards, > Anton > > On Wed, Jul 11, 2018 at 5:00 PM Mikhail Gryzykhin <mig...@google.com> > wrote: > >> @Yifan Zou <yifan...@google.com> >> >> I believe that we should test-drive the system with tickets + PR first >> and decide on email notification later. We already have tests failure >> emails sent to commits@, I believe most people filter out or not signed >> up for that list though. >> >> It creates only one ticket, and keeps it for recurring test failures. >> >> @Andrew Pilloud <apill...@google.com> >> Thank you for the suggestion. I'll add it to design doc. >> >> --Mikhail >> >> >> >> On Wed, Jul 11, 2018 at 4:52 PM Yifan Zou <yifan...@google.com> wrote: >> >>> +1 to Andrew's concerns. Leaving the tickets unassigned will cause the >>> ticket being ignored and no actions being taken. >>> >>> I can see the challenges on ticket assignment. Like Mikhail mentioned, >>> the plugin does not support dynamic assignments. We have to implement >>> custom script to determine the assignees and do some tricks to the jenkins >>> job. Also, the post-commits tests usually cover tons of stuffs that it is >>> difficult to find which part was broken and ask the right person to look >>> into within the Auto JIRA process. Some naive thoughts: Are we able to send >>> emails to the dev@ to ask people to take care of the JIRA issues? Are >>> we able to find component leads and ask them triage the test failure >>> tickets? >>> >>> Another nitpick comment. Does the jenkins job file the JIRA issue in >>> every test failure? Sometimes the test continuously fails in a time period >>> due to the same reason. In this case, we will get some duplicate issues >>> filed by Jenkins. I think it could be better if we can avoid filing issues >>> if the previous one has not been resolved. >>> >>> Thanks. >>> Yifan >>> >>> >>> On Wed, Jul 11, 2018 at 4:37 PM Andrew Pilloud <apill...@google.com> >>> wrote: >>> >>>> That sounds great. You should add this detail to the doc. >>>> >>>> On Wed, Jul 11, 2018 at 4:29 PM Mikhail Gryzykhin <mig...@google.com> >>>> wrote: >>>> >>>>> We already have component for this purpose: "test-failures". All >>>>> tickets created will go to that component. As an option, we can add link >>>>> to >>>>> view list of open JIRA tickets to PR template. >>>>> >>>>> We also would want to create graph on dashboard with amount of >>>>> unassigned and assigned bugs. >>>>> >>>>> I believe that we can also add counter of unassigned bugs to PR >>>>> template. This way it will be easier for everyone to know when there's >>>>> some >>>>> tests issue not attended. >>>>> >>>>> --Mikhail >>>>> >>>>> >>>>> On Wed, Jul 11, 2018 at 4:24 PM Andrew Pilloud <apill...@google.com> >>>>> wrote: >>>>> >>>>>> So it sounds like you will want to create a component for untriaged >>>>>> issues so they are easy to find. I like the idea of distributing the work >>>>>> of triaging post commit failures to new PR authors as a condition of >>>>>> merging. I feel like we will just be filling JIRA with spam if the issues >>>>>> are automatically created without a plan for triage. >>>>>> >>>>>> Andrew >>>>>> >>>>>> On Wed, Jul 11, 2018 at 4:12 PM Rui Wang <ruw...@google.com> wrote: >>>>>> >>>>>>> Maybe this is also a good thread to start the discussion that if we >>>>>>> want to enforce postcommit test for every PR. >>>>>>> >>>>>>> Can we afford the cost of longer waiting time to catch potential >>>>>>> bugs? >>>>>>> >>>>>>> -Rui >>>>>>> >>>>>>> On Wed, Jul 11, 2018 at 4:04 PM Mikhail Gryzykhin <mig...@google.com> >>>>>>> wrote: >>>>>>> >>>>>>>> That's a valid point. >>>>>>>> >>>>>>>> Unfortunately, the JiraTestResultReporter plugin does not have >>>>>>>> features to dynamically assign owners. Additionally, I don't think it >>>>>>>> is >>>>>>>> always easy to find proper owner for post-commit tests at first glance, >>>>>>>> since they usually cover broad specter of issues. >>>>>>>> >>>>>>>> My assumption is that we need someone to triage new issues. >>>>>>>> >>>>>>>> Ideally, any contributor, who sees failing test, should check >>>>>>>> unassigned tickets and either do triage, or assign them to someone who >>>>>>>> can. >>>>>>>> I strongly encourage this approach. >>>>>>>> >>>>>>>> We have couple other ready-made options to consider: >>>>>>>> 1. We can configure JIRA component owner who would be assigned to >>>>>>>> created tickets. >>>>>>>> 2. JiraTestReporterPlugin can assign tickets to specific user. This >>>>>>>> is configured per Jenkins job. We can utilize this if someone >>>>>>>> volunteers. >>>>>>>> 3. Dynamic assignment will most likely require custom solution. >>>>>>>> >>>>>>>> --Mikhail >>>>>>>> >>>>>>>> >>>>>>>> On Wed, Jul 11, 2018 at 3:34 PM Andrew Pilloud <apill...@google.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi Mikhail, >>>>>>>>> >>>>>>>>> I like the proposal! Hopefully this can replace the constant >>>>>>>>> stream of build failure emails. I noticed one detail seems to be >>>>>>>>> missing: >>>>>>>>> How will new issues be assigned to the proper owner? Will the tool do >>>>>>>>> this >>>>>>>>> automatically or will we need someone to triage new issues? >>>>>>>>> >>>>>>>>> Andrew >>>>>>>>> >>>>>>>>> On Wed, Jul 11, 2018 at 3:07 PM Mikhail Gryzykhin < >>>>>>>>> mig...@google.com> wrote: >>>>>>>>> >>>>>>>>>> Hi everyone, >>>>>>>>>> >>>>>>>>>> I want to add an automatic JIRA tickets creation for failing >>>>>>>>>> post-commit tests. >>>>>>>>>> >>>>>>>>>> I wrote up design proposal doc with more details on this: >>>>>>>>>> >>>>>>>>>> https://docs.google.com/document/d/1kpsTy0sYJkLqlZvkPalDkqzBbpu-Wug0z-oWIVPo6UI >>>>>>>>>> >>>>>>>>>> Quick summary: >>>>>>>>>> I suggest to utilize JiraTestResultReporter plugin. >>>>>>>>>> Since this plugin is not installed on our Jenkins yet, we have to >>>>>>>>>> request to Infra team to add it. >>>>>>>>>> >>>>>>>>>> Please, comment if this approach sounds good to you. >>>>>>>>>> >>>>>>>>>> Best regards, >>>>>>>>>> --Mikhail >>>>>>>>>> >>>>>>>>>>
