Thanks chai! Shall we remove the awaiting testing state? To me it doesn't really provide any value and will just produce constant state switches and clutter in the PR. I would say that the state should not change while CI is still running until it reached a conclusion - whether that's one failed or all passed.
-Marco On Thu, Oct 15, 2020, 00:15 Chaitanya Bapat <[email protected]> wrote: > Update: PR Status Label Bot is functional. > > You'll see your PRs getting labels such as > - pr-awaiting-testing [When CI is kick-started for every commit] > - pr-work-in-progress [When CI has failed or PR is marked draft or WIP in > title] > - pr-pending-review [When CI succeeded & no review] > - pr-pending-merge [When CI succeeds & 2 Apache MXNet Committers approve] > - pr-awaiting-response [When CI succeeded but reviewer request changes] > > PR: https://github.com/apache/incubator-mxnet-ci/pull/27 > Followup PR to enable add/remove label : > https://github.com/apache/incubator-mxnet-ci/pull/33 > > > On Sun, 2 Aug 2020 at 13:56, Chaitanya Bapat <[email protected]> wrote: > > > Yes. It has been disabled. > > > > On Sun, 2 Aug 2020 at 13:14, Marco de Abreu <[email protected]> > > wrote: > > > >> Great. As I said, please make sure that the bot is disabled until it has > >> been reviewed and approved. > >> > >> -Marco > >> > >> On Sun, Aug 2, 2020, 10:12 PM Chaitanya Bapat <[email protected]> > >> wrote: > >> > >> > Thanks. webhook is setup! > >> > > >> > On Sun, 2 Aug 2020 at 02:43, Marco de Abreu <[email protected]> > >> > wrote: > >> > > >> > > Ticket created: https://issues.apache.org/jira/browse/INFRA-20624 > >> > > > >> > > -Marco > >> > > > >> > > On Sun, Aug 2, 2020, 7:43 AM Chaitanya Bapat <[email protected]> > >> > wrote: > >> > > > >> > > > Right. > >> > > > I've sent you the email on your personal email address. > >> > > > Thanks for helping setup the github webhook with Apache Infra. > >> > > > > >> > > > Cheers > >> > > > Chai > >> > > > > >> > > > On Sat, 1 Aug 2020 at 03:25, Marco de Abreu < > >> [email protected]> > >> > > > wrote: > >> > > > > >> > > > > Maybe we should just remove the pr-awaiting-testing state > >> entirely. > >> > > > Either > >> > > > > it's WIP or CI passed. > >> > > > > > >> > > > > -Marco > >> > > > > > >> > > > > On Sat, Aug 1, 2020 at 12:21 PM Marco de Abreu < > >> > > [email protected]> > >> > > > > wrote: > >> > > > > > >> > > > > > Hi, > >> > > > > > > >> > > > > > I generally like the idea, thanks for improving the user > >> experience > >> > > for > >> > > > > > the project, Chai! One thing that would be important to me is > >> > > avoiding > >> > > > a > >> > > > > > constant shifting between the states. The PR shouldn't switch > to > >> > > > > > "awaiting-testing" every single time CI is retriggered but > >> rather > >> > > have > >> > > > > some > >> > > > > > consistency. But that's something we can also tweak as we go. > >> > > > > > > >> > > > > > Before the bot is run, a small heads up would be nice. Just > >> > running a > >> > > > > > batch job on the repository - especially using others' > >> credentials > >> > - > >> > > > > > without previous approval is not that nice. Hence, please run > >> your > >> > > > tests > >> > > > > a > >> > > > > > non-intrusive way until the bot has been approved for > >> deployment. > >> > > > > > > >> > > > > > With regards to the ticket, I'm happy to be of assistance. > Just > >> > send > >> > > > the > >> > > > > > secret to my personal email. > >> > > > > > > >> > > > > > Best regards > >> > > > > > Marco > >> > > > > > > >> > > > > > On Fri, Jul 31, 2020 at 9:12 PM Sheng Zha < > [email protected]> > >> > > wrote: > >> > > > > > > >> > > > > >> +1, thanks for driving this. Here's the pseudo code for the > PR > >> > > > labeling > >> > > > > >> for better readability: > >> > > > > >> > >> > > > > >> if WIP in title or PR is draft or CI failed: > >> > > > > >> pr-work-in-progress > >> > > > > >> elif CI has not started yet or CI is in progress: > >> > > > > >> pr-awaiting-testing > >> > > > > >> else: # CI passed checks > >> > > > > >> if pr has at least one approval and no request changes: > >> > > > > >> pr-awaiting-merge > >> > > > > >> elif pr has no review or all reviews have been > >> > > > > dismissed/re-requested: > >> > > > > >> pr-awaiting-review > >> > > > > >> else: # pr has a review that hasn't been dismissed yet no > >> > > approval > >> > > > > >> pr-awaiting-response > >> > > > > >> > >> > > > > >> Best, > >> > > > > >> Sheng > >> > > > > >> > >> > > > > >> On 2020/07/31 18:08:34, Chaitanya Bapat < > [email protected]> > >> > > wrote: > >> > > > > >> > Hello MXNet community, > >> > > > > >> > > >> > > > > >> > Banking on the success of mxnet-bot that has been > triggering > >> CI > >> > > > > >> > successfully for the past 3 months, Sheng proposed an idea > >> for > >> > > > > >> automatic PR > >> > > > > >> > labeling. I am working on the implementation of the same. > >> > > > > >> > > >> > > > > >> > *What's the idea?* > >> > > > > >> > An automated way of labeling PRs based on the status of CI. > >> > > > > >> > > >> > > > > >> > *Why?* > >> > > > > >> > > >> > > > > >> > This serves dual purpose > >> > > > > >> > > >> > > > > >> > - Automates the labeling [currently manual] > >> > > > > >> > - Allows contributors to prioritize review of PRs. > >> > > > > >> > > >> > > > > >> > *How?* > >> > > > > >> > 1. Github webhook would send status update events to the > >> Lambda > >> > > > > function > >> > > > > >> > 2. Lambda would then label the CI based on following > >> conditions > >> > > > > >> > > >> > > > > >> > > >> > > > > >> > > >> > > > > >> > > >> > > > > >> > > >> > > > > >> > > >> > > > > >> > > >> > > > > >> > > >> > > > > >> > > >> > > > > >> > > >> > > > > >> > > >> > > > > >> > *if WIP in title or PR is draft or CI failed: > >> > > > > >> pr-work-in-progresselif CI > >> > > > > >> > has not started yet or CI is in progress: > >> > > > pr-awaiting-testingelse: > >> > > > > # > >> > > > > >> CI > >> > > > > >> > passed checks if pr has at least one approval and no > >> request > >> > > > > changes: > >> > > > > >> > pr-awaiting-merge elif pr has no review or all > reviews > >> > have > >> > > > > been > >> > > > > >> > dismissed/re-requested: pr-awaiting-review else: > # > >> pr > >> > > has > >> > > > a > >> > > > > >> > review that hasn't been dismissed yet no approval > >> > > > > >> > pr-awaiting-response* > >> > > > > >> > > >> > > > > >> > To that end, a POC was done to test if it's possible & > >> result: > >> > it > >> > > > > works. > >> > > > > >> > Results of 1 successful run on Prod for > >> apache/incubator-mxnet > >> > > repo > >> > > > > >> > > >> > > > > >> > Open PRs : 207 > >> > > > > >> > PRs labeled : 18 > >> > > > > >> > PRs already labeled : 38 > >> > > > > >> > PRs with unknown jobs : 2 > >> > > > > >> > PRs with status failed : 149 > >> > > > > >> > > >> > > > > >> > Lambda function labeled 18 PRs that had passed all the > tests. > >> > > > > >> > > >> > > > > >> > PR for POC: > >> > https://github.com/apache/incubator-mxnet-ci/pull/27 > >> > > > > >> > However, Marco made a valid point regarding leveraging > Github > >> > > > Webhooks > >> > > > > >> > [instead of Jenkins] for retrieving status of the PR. > >> > > > > >> > > >> > > > > >> > *What's required?* > >> > > > > >> > Thus, to get this feature functional, I'd need a separate > >> Github > >> > > > > webhook > >> > > > > >> > that sends information related to PR Status. Just like the > >> > webhook > >> > > > > >> request > >> > > > > >> > for CI Bot, I can send the details over private email to > >> Marco's > >> > > > > >> personal > >> > > > > >> > email address & then he can create a ticket with Apache > >> Infra. > >> > > > > >> > > >> > > > > >> > Thanks to Sheng & Marco for guidance & assistance on this. > >> > > > > >> > > >> > > > > >> > Thanks, > >> > > > > >> > Chai > >> > > > > >> > > >> > > > > >> > -- > >> > > > > >> > *Chaitanya Prakash Bapat* > >> > > > > >> > *+1 (973) 953-6299* > >> > > > > >> > > >> > > > > >> > [image: https://www.linkedin.com//in/chaibapat25] > >> > > > > >> > <https://github.com/ChaiBapchya>[image: > >> > > > > >> https://www.facebook.com/chaibapat] > >> > > > > >> > <https://www.facebook.com/chaibapchya>[image: > >> > > > > >> > https://twitter.com/ChaiBapchya] < > >> > https://twitter.com/ChaiBapchya > >> > > > > >> >[image: > >> > > > > >> > https://www.linkedin.com//in/chaibapat25] > >> > > > > >> > <https://www.linkedin.com//in/chaibapchya/> > >> > > > > >> > > >> > > > > >> > >> > > > > > > >> > > > > > >> > > > > >> > > > > >> > > > -- > >> > > > *Chaitanya Prakash Bapat* > >> > > > *+1 (973) 953-6299* > >> > > > > >> > > > [image: https://www.linkedin.com//in/chaibapat25] > >> > > > <https://github.com/ChaiBapchya>[image: > >> > > https://www.facebook.com/chaibapat > >> > > > ] > >> > > > <https://www.facebook.com/chaibapchya>[image: > >> > > > https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya > >> > > >[image: > >> > > > https://www.linkedin.com//in/chaibapat25] > >> > > > <https://www.linkedin.com//in/chaibapchya/> > >> > > > > >> > > > >> > > >> > > >> > -- > >> > *Chaitanya Prakash Bapat* > >> > *+1 (973) 953-6299* > >> > > >> > [image: https://www.linkedin.com//in/chaibapat25] > >> > <https://github.com/ChaiBapchya>[image: > >> https://www.facebook.com/chaibapat > >> > ] > >> > <https://www.facebook.com/chaibapchya>[image: > >> > https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya > >> >[image: > >> > https://www.linkedin.com//in/chaibapat25] > >> > <https://www.linkedin.com//in/chaibapchya/> > >> > > >> > > > > > > -- > > *Chaitanya Prakash Bapat* > > *+1 (973) 953-6299* > > > > [image: https://www.linkedin.com//in/chaibapat25] > > <https://github.com/ChaiBapchya>[image: > > https://www.facebook.com/chaibapat] < > https://www.facebook.com/chaibapchya>[image: > > https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya > >[image: > > https://www.linkedin.com//in/chaibapat25] > > <https://www.linkedin.com//in/chaibapchya/> > > > > > -- > *Chaitanya Prakash Bapat* > *+1 (973) 953-6299* > > [image: https://www.linkedin.com//in/chaibapat25] > <https://github.com/ChaiBapchya>[image: https://www.facebook.com/chaibapat > ] > <https://www.facebook.com/chaibapchya>[image: > https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya>[image: > https://www.linkedin.com//in/chaibapat25] > <https://www.linkedin.com//in/chaibapchya/> >
