@Yifan Zou <yifan...@google.com> I believe that we should test-drive the system with tickets + PR first and decide on email notification later. We already have tests failure emails sent to commits@, I believe most people filter out or not signed up for that list though.
It creates only one ticket, and keeps it for recurring test failures. @Andrew Pilloud <apill...@google.com> Thank you for the suggestion. I'll add it to design doc. --Mikhail On Wed, Jul 11, 2018 at 4:52 PM Yifan Zou <yifan...@google.com> wrote: > +1 to Andrew's concerns. Leaving the tickets unassigned will cause the > ticket being ignored and no actions being taken. > > I can see the challenges on ticket assignment. Like Mikhail mentioned, the > plugin does not support dynamic assignments. We have to implement custom > script to determine the assignees and do some tricks to the jenkins job. > Also, the post-commits tests usually cover tons of stuffs that it is > difficult to find which part was broken and ask the right person to look > into within the Auto JIRA process. Some naive thoughts: Are we able to send > emails to the dev@ to ask people to take care of the JIRA issues? Are we > able to find component leads and ask them triage the test failure tickets? > > Another nitpick comment. Does the jenkins job file the JIRA issue in every > test failure? Sometimes the test continuously fails in a time period due to > the same reason. In this case, we will get some duplicate issues filed by > Jenkins. I think it could be better if we can avoid filing issues if the > previous one has not been resolved. > > Thanks. > Yifan > > > On Wed, Jul 11, 2018 at 4:37 PM Andrew Pilloud <apill...@google.com> > wrote: > >> That sounds great. You should add this detail to the doc. >> >> On Wed, Jul 11, 2018 at 4:29 PM Mikhail Gryzykhin <mig...@google.com> >> wrote: >> >>> We already have component for this purpose: "test-failures". All tickets >>> created will go to that component. As an option, we can add link to view >>> list of open JIRA tickets to PR template. >>> >>> We also would want to create graph on dashboard with amount of >>> unassigned and assigned bugs. >>> >>> I believe that we can also add counter of unassigned bugs to PR >>> template. This way it will be easier for everyone to know when there's some >>> tests issue not attended. >>> >>> --Mikhail >>> >>> >>> On Wed, Jul 11, 2018 at 4:24 PM Andrew Pilloud <apill...@google.com> >>> wrote: >>> >>>> So it sounds like you will want to create a component for untriaged >>>> issues so they are easy to find. I like the idea of distributing the work >>>> of triaging post commit failures to new PR authors as a condition of >>>> merging. I feel like we will just be filling JIRA with spam if the issues >>>> are automatically created without a plan for triage. >>>> >>>> Andrew >>>> >>>> On Wed, Jul 11, 2018 at 4:12 PM Rui Wang <ruw...@google.com> wrote: >>>> >>>>> Maybe this is also a good thread to start the discussion that if we >>>>> want to enforce postcommit test for every PR. >>>>> >>>>> Can we afford the cost of longer waiting time to catch potential bugs? >>>>> >>>>> -Rui >>>>> >>>>> On Wed, Jul 11, 2018 at 4:04 PM Mikhail Gryzykhin <mig...@google.com> >>>>> wrote: >>>>> >>>>>> That's a valid point. >>>>>> >>>>>> Unfortunately, the JiraTestResultReporter plugin does not have >>>>>> features to dynamically assign owners. Additionally, I don't think it is >>>>>> always easy to find proper owner for post-commit tests at first glance, >>>>>> since they usually cover broad specter of issues. >>>>>> >>>>>> My assumption is that we need someone to triage new issues. >>>>>> >>>>>> Ideally, any contributor, who sees failing test, should check >>>>>> unassigned tickets and either do triage, or assign them to someone who >>>>>> can. >>>>>> I strongly encourage this approach. >>>>>> >>>>>> We have couple other ready-made options to consider: >>>>>> 1. We can configure JIRA component owner who would be assigned to >>>>>> created tickets. >>>>>> 2. JiraTestReporterPlugin can assign tickets to specific user. This >>>>>> is configured per Jenkins job. We can utilize this if someone volunteers. >>>>>> 3. Dynamic assignment will most likely require custom solution. >>>>>> >>>>>> --Mikhail >>>>>> >>>>>> >>>>>> On Wed, Jul 11, 2018 at 3:34 PM Andrew Pilloud <apill...@google.com> >>>>>> wrote: >>>>>> >>>>>>> Hi Mikhail, >>>>>>> >>>>>>> I like the proposal! Hopefully this can replace the constant stream >>>>>>> of build failure emails. I noticed one detail seems to be missing: How >>>>>>> will new issues be assigned to the proper owner? Will the tool do this >>>>>>> automatically or will we need someone to triage new issues? >>>>>>> >>>>>>> Andrew >>>>>>> >>>>>>> On Wed, Jul 11, 2018 at 3:07 PM Mikhail Gryzykhin <mig...@google.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi everyone, >>>>>>>> >>>>>>>> I want to add an automatic JIRA tickets creation for failing >>>>>>>> post-commit tests. >>>>>>>> >>>>>>>> I wrote up design proposal doc with more details on this: >>>>>>>> >>>>>>>> https://docs.google.com/document/d/1kpsTy0sYJkLqlZvkPalDkqzBbpu-Wug0z-oWIVPo6UI >>>>>>>> >>>>>>>> Quick summary: >>>>>>>> I suggest to utilize JiraTestResultReporter plugin. >>>>>>>> Since this plugin is not installed on our Jenkins yet, we have to >>>>>>>> request to Infra team to add it. >>>>>>>> >>>>>>>> Please, comment if this approach sounds good to you. >>>>>>>> >>>>>>>> Best regards, >>>>>>>> --Mikhail >>>>>>>> >>>>>>>>
