Re: [FYI] CI Infra issues (in both GitHub Action and Jenkins)

2021-01-09 Thread Hyukjin Kwon
To share the update about GitHub Actions: - I am informed that having more resources is now being discussed in org level. Hopefully we get the situation better. - Duplicated workflows will be canceled to save resources, see https://github.com/apache/spark/pull/31104 cc @Holden Karau @Xiao Li

Re: [FYI] CI Infra issues (in both GitHub Action and Jenkins)

2021-01-08 Thread Hyukjin Kwon
For GitHub resources of ASF repo, I have been contacting GitHub to address the issue few days ago. This is not a repo level problem cc @Sean Owen . ASF organisation in GitHub has already too many repos, and we should have a way to increase the limit, or set the separare limit specifically for the

Re: [FYI] CI Infra issues (in both GitHub Action and Jenkins)

2021-01-08 Thread shane knapp ☠
no, i don't think that'd be a good idea... adding additional dependencies to our cluster won't scale one bit. On Fri, Jan 8, 2021 at 2:16 PM Dongjoon Hyun wrote: > BTW, Shane, do you think we can utilize some of UCB machines as GitHub > Action runners? > > Bests, > Dongjoon. > > On Fri, Jan 8,

Re: [FYI] CI Infra issues (in both GitHub Action and Jenkins)

2021-01-08 Thread shane knapp ☠
hmm, the ubuntu16 machines are acting up. i pinned the sbt master builds to ubuntu20 and they're happily building while i investigate wtf is up. On Fri, Jan 8, 2021 at 2:15 PM Dongjoon Hyun wrote: > The followings? > > >

Re: [FYI] CI Infra issues (in both GitHub Action and Jenkins)

2021-01-08 Thread Dongjoon Hyun
BTW, Shane, do you think we can utilize some of UCB machines as GitHub Action runners? Bests, Dongjoon. On Fri, Jan 8, 2021 at 2:14 PM Dongjoon Hyun wrote: > The followings? > > >

Re: [FYI] CI Infra issues (in both GitHub Action and Jenkins)

2021-01-08 Thread Dongjoon Hyun
The followings? https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/job/spark-master-test-sbt-hadoop-3.2/1836/console https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/job/spark-master-test-sbt-hadoop-2.7/1887/console On Fri, Jan 8, 2021 at 2:13

Re: [FYI] CI Infra issues (in both GitHub Action and Jenkins)

2021-01-08 Thread shane knapp ☠
> > 1. Jenkins machines start to fail with the following recently. > (master branch) > > Python versions prior to 3.6 are not supported. > Build step 'Execute shell' marked build as failure > > examples please? -- Shane Knapp Computer Guy / Voice of Reason UC Berkeley EECS Research /

[FYI] CI Infra issues (in both GitHub Action and Jenkins)

2021-01-08 Thread Dongjoon Hyun
Hi, All. There are two issues currently. 1. Jenkins machines start to fail with the following recently. (master branch) Python versions prior to 3.6 are not supported. Build step 'Execute shell' marked build as failure 2. Lack of GitHub Action backend machines. There is a JIRA