Yep. Now we just need credits :) On Tue, Oct 13, 2020 at 5:30 PM Kaxil Naik <[email protected]> wrote:
> That's ace, we should go ahead with self-hosted runners then. > > On Tue, Oct 13, 2020 at 4:06 PM Ash Berlin-Taylor <[email protected]> wrote: > >> Confirmed, we *can* do it - Arrow has done it already >> https://issues.apache.org/jira/browse/INFRA-19875 >> >> But lets have a think on how to not be a bot net :) >> >> On Oct 13 2020, at 3:59 pm, Ash Berlin-Taylor <[email protected]> wrote: >> >> I've spoken to a few members of ASF Infra directly, and they are just >> confirming but they are okay with the idea of us adding self hosted runners >> to our repo, and also okay that we can manage those nodes ourselves. Should >> get final confirmation today. >> >> I wanted to double check that we could use the credits before we get >> anyone to stump up the VMs/credits etc. >> >> -ash >> >> On Oct 13 2020, at 2:16 pm, Jarek Potiuk <[email protected]> >> wrote: >> >> This is also a slight problem as mentioned in the build@ thread: >> https://lists.apache.org/thread.html/r1708881f52adbdae722afb8fea16b23325b739b254b60890e72375e1%40%3Cbuilds.apache.org%3E >> - >> managing hosting runners has to be done through infrastructure and they are >> not really responsive recently (I have tickets waiting for weeks now). >> >> But as I've learned recently that we can manage our own secrets via API >> without INFRA (and completely legitimately according to GitHub >> documentation), maybe hosted runners will be also possible to self-manage :D >> >> J. >> >> On Tue, Oct 13, 2020 at 2:22 PM Ash Berlin-Taylor <[email protected]> wrote: >> >> I've thought about private/self-hosted runners, and I think long term >> that's the way to go to alievate our CI bottlenecks. >> >> There's a bit of work we need to do around security of builds - as >> mentioned here >> https://docs.github.com/en/free-pro-team@latest/actions/hosting-your-own-runners/about-self-hosted-runners#self-hosted-runner-security-with-public-repositories >> >> > We recommend that you do not use self-hosted runners with public >> repositories. >> > >> > Forks of your public repository can potentially run dangerous code on >> your self-hosted runner machine by creating a pull request that executes >> the code in a workflow. >> > >> > This is not an issue with GitHub-hosted runners because each >> GitHub-hosted runner is always a clean isolated virtual machine, and it is >> destroyed at the end of the job execution. >> >> So we'd need to dos something similar. >> >> All for this and happy to help out once 2.0 is out (or at least once it >> starts to quieten down) >> >> -ash >> >> On Oct 13 2020, at 1:12 pm, Jarek Potiuk <[email protected]> >> wrote: >> >> Hello Aizhamal, Everyone, >> >> We've had some problems recently with concurrency for Github Actions and >> suggested solution for now is to use self-hosted runners (This is suggested >> by GitHub Support) >> >> I made some comments in the issue here: >> >> https://github.com/apache/airflow/issues/11496 >> >> And also opened build@ discussion >> https://lists.apache.org/thread.html/r1708881f52adbdae722afb8fea16b23325b739b254b60890e72375e1%40%3Cbuilds.apache.org%3E >> and >> opened an accompanying ticket in JIRA: >> https://issues.apache.org/jira/projects/INFRA/issues/INFRA-20978 >> >> Regardless from those discussions, It would be great if we come back to >> the idea of Google Donating some credits to Apache Airlfow to setup their >> own runners. >> >> We have not used them last time when GitLab did not manage to implement >> the needed fork support (they have not implemented it till NOW for more >> than 1.5 year!) but with GitHub I am quite certain we can switch and start >> using such runners pretty much immediately if we had some credits. >> >> Or maybe some other companies could donate some credits to us ? >> >> J. >> >> >> >> >> -- >> >> Jarek Potiuk >> Polidea <https://www.polidea.com/> | Principal Software Engineer >> >> M: +48 660 796 129 <+48660796129> >> [image: Polidea] <https://www.polidea.com/> >> >> >> >> -- >> >> Jarek Potiuk >> Polidea <https://www.polidea.com/> | Principal Software Engineer >> >> M: +48 660 796 129 <+48660796129> >> [image: Polidea] <https://www.polidea.com/> >> >> -- Jarek Potiuk Polidea <https://www.polidea.com/> | Principal Software Engineer M: +48 660 796 129 <+48660796129> [image: Polidea] <https://www.polidea.com/>
