CI woes

2019-05-16 Thread Per da Silva
Dear MXNet community, The credential rotaion and clean up had the undesirable side-effect of triggering all of the PR jobs. This caused Jenkins to overflow and CI got wonky. After some computer hacking we've managed to get everything back to normal. Unfortunately, this may have reset the state of

Re: Jenkins possibly compromised

2019-05-16 Thread Marco de Abreu
Hey again, the webhook has successfully been re-activated and thus all PRs should be processed again. As part of the process, we had to retrigger all jobs which will re-evaluate all PRs and make a huge queue on our side. Autoscaling will kick in and try to chew through all jobs, but it will take a

Jenkins possibly compromised

2019-05-16 Thread Marco de Abreu
Hello dev@, we noticed some fishy logs on our Jenkins and are afraid that a successful attack might have happened. Thus, we are taking security precautions and rotating all credentials and revoking active session. As part of this process, we have to rotate the GitHub webhook secret. This update r