Re: MXNet Bot Demo
Thank you Chaitanya and Marco for helping the MXNet community. On Mon, Mar 23, 2020 at 12:56 PM Marco de Abreu wrote: > Sure, already done. > > -Marco > > On Mon, Mar 23, 2020 at 8:53 PM Chaitanya Bapat > wrote: > > > Hello, > > Update: Apache Infra Ticket for MXNet Bot > > Thanks once again, Marco for opening the ticket. But turns out, Apache > > Infra folks closed it stating: "Security concerns around allowing unknown > > person to submit PR and run our hardware". Furthermore, it goes onto > state > > that bot circumvents the dependence on Jenkins Admins which is like > solving > > a problem that doesn't exist. > > > > I sense there is some confusion in the communication (maybe on my part). > It > > turns out the security concerns aren't actually correct. > > > > 1. Unknown person can submit a PR (before & after bot proposal), and run > > our hardware (pre as well as post bot). > > 2. Code should be reviewed by somebody with an ICLA on file. This doesn't > > change either. Prior to merging a PR, code has to be approved by a > > committer just like before. > > Overall it looks like the job of the bot isn't clear to folks in Apache > > Infra. Bot simply is a means for triggering CI (which could be done > > manually by Log In to Jenkins -> PR -> Job -> Build) and doesn't quite > > tweak with merging procedure. Yes, only addition is now unknown person > (PR > > Author) can trigger CI with a message (but that was possible anyway by > > pushing a commit. Bot just prevents users from pushing empty commits and > > building entire suite). > > > > As can be seen from last 10 open PRs as of Monday 23rd March, 12pm PT > most > > PRs fail on 1/2 jobs. In such a scenario, the proposed MXNet bot would > come > > in handy for just invoking CI on that specific build (instead of a > > non-committer PR Author to push empty commit : hurting on the resource, > > time & cost considerations apart from undesirable dev experience) > > > > @Marco Since I am a non-committer, I guess these 2 clarifications need to > > be conveyed to the Apache Infra by someone with Committer access. > > > > What do you think? > > > > Thanks, > > Chai > > > > On Sat, 21 Mar 2020 at 16:08, Marco de Abreu > > wrote: > > > > > Hello, > > > > > > the ticket has been created: > > > https://issues.apache.org/jira/browse/INFRA-20005 > > > > > > Best regards, > > > Marco > > > > > > On Thu, Mar 19, 2020 at 11:49 PM Marco de Abreu < > marco.g.ab...@gmail.com > > > > > > wrote: > > > > > > > Sounds like a good plan! > > > > > > > > Please send me the URL (please make sure it's backed by DNS and not > > just > > > > the gateway URL) of the webhook handler, GitHub events you're > > interested > > > in > > > > and the shared secret in a private email to my personal email > address. > > I > > > > will then create the ticket with Apache infra. > > > > > > > > -Marco > > > > > > > > Chaitanya Bapat schrieb am Do., 19. März > 2020, > > > > 23:07: > > > > > > > >> @Marco Alright, it makes total sense to test out the Bot feature > > > alongside > > > >> auto-trigger as a transition. > > > >> > > > >> Path Forward: > > > >> 1. Setup MXNet Bot on apache/incubator-mxnet repo (GitHub WebHook > and > > > >> Infra) > > > >> 2. We don't turn off automatic trigger of PR builds for now. > > > >> 3. Hopefully, bot is used by developers to trigger specific jobs > > > >> 4. Later on (say around April 20), let's discuss the possibility of > > > >> switching off auto-trigger (with appropriate data) if it makes > sense. > > > >> Thanks Marco for volunteering to help enable the web hook on > > > >> apache/incubator-mxnet. Let me know if we can sync up on Slack > channel > > > to > > > >> get the ball rolling. > > > >> > > > >> Thanks once again for the entire community to step in and help try > out > > > >> this > > > >> Bot. > > > >> Chai > > > >> > > > >> On Wed, 18 Mar 2020 at 17:07, Marco de Abreu < > marco.g.ab...@gmail.com > > > > > > >> wrote: > > > >> > > > >> > Hi, that's correct. But as stated previously, it's not an option > to > > > >> remove > > > >> > the hook. For now, I'd like to see how the system behaves while > it's > > > >> > optional. Later on, we can talk about revisiting this decision. > But > > to > > > >> me > > > >> > it's not an option to deploy an entirely new system and approach > > > without > > > >> > having a transition or even a timeframe in which we are able to > fall > > > >> back. > > > >> > > > > >> > I'm happy to support the deployment of the bot and add an > additional > > > >> > webhook to enable it's functionality to support selective > triggering > > > by > > > >> PR > > > >> > authors and committers, but I will not support the disabling of > > > >> automatic > > > >> > triggering of branches or PRs. > > > >> > > > > >> > -Marco > > > >> > > > > >> > Chaitanya Bapat schrieb am Mi., 18. März > > 2020, > > > >> > 21:00: > > > >> > > > > >> > > Hey Marco, > > > >> > > > > > >> > > I thought currently every commit on PR and master
Re: MXNet Bot Demo
Sure, already done. -Marco On Mon, Mar 23, 2020 at 8:53 PM Chaitanya Bapat wrote: > Hello, > Update: Apache Infra Ticket for MXNet Bot > Thanks once again, Marco for opening the ticket. But turns out, Apache > Infra folks closed it stating: "Security concerns around allowing unknown > person to submit PR and run our hardware". Furthermore, it goes onto state > that bot circumvents the dependence on Jenkins Admins which is like solving > a problem that doesn't exist. > > I sense there is some confusion in the communication (maybe on my part). It > turns out the security concerns aren't actually correct. > > 1. Unknown person can submit a PR (before & after bot proposal), and run > our hardware (pre as well as post bot). > 2. Code should be reviewed by somebody with an ICLA on file. This doesn't > change either. Prior to merging a PR, code has to be approved by a > committer just like before. > Overall it looks like the job of the bot isn't clear to folks in Apache > Infra. Bot simply is a means for triggering CI (which could be done > manually by Log In to Jenkins -> PR -> Job -> Build) and doesn't quite > tweak with merging procedure. Yes, only addition is now unknown person (PR > Author) can trigger CI with a message (but that was possible anyway by > pushing a commit. Bot just prevents users from pushing empty commits and > building entire suite). > > As can be seen from last 10 open PRs as of Monday 23rd March, 12pm PT most > PRs fail on 1/2 jobs. In such a scenario, the proposed MXNet bot would come > in handy for just invoking CI on that specific build (instead of a > non-committer PR Author to push empty commit : hurting on the resource, > time & cost considerations apart from undesirable dev experience) > > @Marco Since I am a non-committer, I guess these 2 clarifications need to > be conveyed to the Apache Infra by someone with Committer access. > > What do you think? > > Thanks, > Chai > > On Sat, 21 Mar 2020 at 16:08, Marco de Abreu > wrote: > > > Hello, > > > > the ticket has been created: > > https://issues.apache.org/jira/browse/INFRA-20005 > > > > Best regards, > > Marco > > > > On Thu, Mar 19, 2020 at 11:49 PM Marco de Abreu > > > wrote: > > > > > Sounds like a good plan! > > > > > > Please send me the URL (please make sure it's backed by DNS and not > just > > > the gateway URL) of the webhook handler, GitHub events you're > interested > > in > > > and the shared secret in a private email to my personal email address. > I > > > will then create the ticket with Apache infra. > > > > > > -Marco > > > > > > Chaitanya Bapat schrieb am Do., 19. März 2020, > > > 23:07: > > > > > >> @Marco Alright, it makes total sense to test out the Bot feature > > alongside > > >> auto-trigger as a transition. > > >> > > >> Path Forward: > > >> 1. Setup MXNet Bot on apache/incubator-mxnet repo (GitHub WebHook and > > >> Infra) > > >> 2. We don't turn off automatic trigger of PR builds for now. > > >> 3. Hopefully, bot is used by developers to trigger specific jobs > > >> 4. Later on (say around April 20), let's discuss the possibility of > > >> switching off auto-trigger (with appropriate data) if it makes sense. > > >> Thanks Marco for volunteering to help enable the web hook on > > >> apache/incubator-mxnet. Let me know if we can sync up on Slack channel > > to > > >> get the ball rolling. > > >> > > >> Thanks once again for the entire community to step in and help try out > > >> this > > >> Bot. > > >> Chai > > >> > > >> On Wed, 18 Mar 2020 at 17:07, Marco de Abreu > > > >> wrote: > > >> > > >> > Hi, that's correct. But as stated previously, it's not an option to > > >> remove > > >> > the hook. For now, I'd like to see how the system behaves while it's > > >> > optional. Later on, we can talk about revisiting this decision. But > to > > >> me > > >> > it's not an option to deploy an entirely new system and approach > > without > > >> > having a transition or even a timeframe in which we are able to fall > > >> back. > > >> > > > >> > I'm happy to support the deployment of the bot and add an additional > > >> > webhook to enable it's functionality to support selective triggering > > by > > >> PR > > >> > authors and committers, but I will not support the disabling of > > >> automatic > > >> > triggering of branches or PRs. > > >> > > > >> > -Marco > > >> > > > >> > Chaitanya Bapat schrieb am Mi., 18. März > 2020, > > >> > 21:00: > > >> > > > >> > > Hey Marco, > > >> > > > > >> > > I thought currently every commit on PR and master triggers CI > > >> > > because > > >> > > a. github webhook points to Jenkins Server > > >> > > b. GH Webhook events trigger builds on Jenkins for all commits to > > any > > >> > > branch in apache/incubator-mxnet > > >> > > may it be master/PR/non-PR > > >> > > Reason: > > >> > > Because all the 3 types of branches are discovered by Jenkins > > (non-PR > > >> > > (including master) and PR) > > >> > > > > >> > > Proposal: Remove GitHub WebHook to Jenkins and replace with
Re: MXNet Bot Demo
Hello, Update: Apache Infra Ticket for MXNet Bot Thanks once again, Marco for opening the ticket. But turns out, Apache Infra folks closed it stating: "Security concerns around allowing unknown person to submit PR and run our hardware". Furthermore, it goes onto state that bot circumvents the dependence on Jenkins Admins which is like solving a problem that doesn't exist. I sense there is some confusion in the communication (maybe on my part). It turns out the security concerns aren't actually correct. 1. Unknown person can submit a PR (before & after bot proposal), and run our hardware (pre as well as post bot). 2. Code should be reviewed by somebody with an ICLA on file. This doesn't change either. Prior to merging a PR, code has to be approved by a committer just like before. Overall it looks like the job of the bot isn't clear to folks in Apache Infra. Bot simply is a means for triggering CI (which could be done manually by Log In to Jenkins -> PR -> Job -> Build) and doesn't quite tweak with merging procedure. Yes, only addition is now unknown person (PR Author) can trigger CI with a message (but that was possible anyway by pushing a commit. Bot just prevents users from pushing empty commits and building entire suite). As can be seen from last 10 open PRs as of Monday 23rd March, 12pm PT most PRs fail on 1/2 jobs. In such a scenario, the proposed MXNet bot would come in handy for just invoking CI on that specific build (instead of a non-committer PR Author to push empty commit : hurting on the resource, time & cost considerations apart from undesirable dev experience) @Marco Since I am a non-committer, I guess these 2 clarifications need to be conveyed to the Apache Infra by someone with Committer access. What do you think? Thanks, Chai On Sat, 21 Mar 2020 at 16:08, Marco de Abreu wrote: > Hello, > > the ticket has been created: > https://issues.apache.org/jira/browse/INFRA-20005 > > Best regards, > Marco > > On Thu, Mar 19, 2020 at 11:49 PM Marco de Abreu > wrote: > > > Sounds like a good plan! > > > > Please send me the URL (please make sure it's backed by DNS and not just > > the gateway URL) of the webhook handler, GitHub events you're interested > in > > and the shared secret in a private email to my personal email address. I > > will then create the ticket with Apache infra. > > > > -Marco > > > > Chaitanya Bapat schrieb am Do., 19. März 2020, > > 23:07: > > > >> @Marco Alright, it makes total sense to test out the Bot feature > alongside > >> auto-trigger as a transition. > >> > >> Path Forward: > >> 1. Setup MXNet Bot on apache/incubator-mxnet repo (GitHub WebHook and > >> Infra) > >> 2. We don't turn off automatic trigger of PR builds for now. > >> 3. Hopefully, bot is used by developers to trigger specific jobs > >> 4. Later on (say around April 20), let's discuss the possibility of > >> switching off auto-trigger (with appropriate data) if it makes sense. > >> Thanks Marco for volunteering to help enable the web hook on > >> apache/incubator-mxnet. Let me know if we can sync up on Slack channel > to > >> get the ball rolling. > >> > >> Thanks once again for the entire community to step in and help try out > >> this > >> Bot. > >> Chai > >> > >> On Wed, 18 Mar 2020 at 17:07, Marco de Abreu > >> wrote: > >> > >> > Hi, that's correct. But as stated previously, it's not an option to > >> remove > >> > the hook. For now, I'd like to see how the system behaves while it's > >> > optional. Later on, we can talk about revisiting this decision. But to > >> me > >> > it's not an option to deploy an entirely new system and approach > without > >> > having a transition or even a timeframe in which we are able to fall > >> back. > >> > > >> > I'm happy to support the deployment of the bot and add an additional > >> > webhook to enable it's functionality to support selective triggering > by > >> PR > >> > authors and committers, but I will not support the disabling of > >> automatic > >> > triggering of branches or PRs. > >> > > >> > -Marco > >> > > >> > Chaitanya Bapat schrieb am Mi., 18. März 2020, > >> > 21:00: > >> > > >> > > Hey Marco, > >> > > > >> > > I thought currently every commit on PR and master triggers CI > >> > > because > >> > > a. github webhook points to Jenkins Server > >> > > b. GH Webhook events trigger builds on Jenkins for all commits to > any > >> > > branch in apache/incubator-mxnet > >> > > may it be master/PR/non-PR > >> > > Reason: > >> > > Because all the 3 types of branches are discovered by Jenkins > (non-PR > >> > > (including master) and PR) > >> > > > >> > > Proposal: Remove GitHub WebHook to Jenkins and replace with GH > >> Webhook to > >> > > Lambda > >> > > But after I remove the github webhook that points to Jenkins : *N**o > >> > commit > >> > > will trigger Jenkins build by default* (as Jenkins wont receive GH > >> > events) > >> > > Only those that Bot deems fit will be triggered (using Jenkins API > >> > invoked > >> > > by Lambda). > >> > > Hence
