Re: Travis CI delays

2019-09-28 Thread Krisztián Szűcs
I'd like to mention that we have the option to reuse the buildbot builders as well. I've put up a PR [1] which runs the exact same builders (without the CUDA ones) what we have in the buildbot setup [2]. The required GitHub Actions configuration is fairly small [3]. [1]: https://github.com/apache/

Re: Travis CI delays

2019-09-28 Thread Sutou Kouhei
id-November. Thanks, -- kou In <479f115c-4640-8fef-9e72-d085d09ea...@python.org> "Re: Travis CI delays" on Sat, 28 Sep 2019 10:11:19 +0200, Antoine Pitrou wrote: > > Is it easy to enable caching (e.g. ccache) with docker-compose and > Github Actions? > >

Re: Travis CI delays

2019-09-28 Thread Antoine Pitrou
Is it easy to enable caching (e.g. ccache) with docker-compose and Github Actions? Regards Antoine. Le 28/09/2019 à 01:36, Francois Saint-Jacques a écrit : > Hello, > > I suggest we tackle https://jira.apache.org/jira/browse/ARROW-5801. > For Rust, that would be > https://jira.apache.org/jir

Re: Travis CI delays

2019-09-27 Thread Francois Saint-Jacques
Hello, I suggest we tackle https://jira.apache.org/jira/browse/ARROW-5801. For Rust, that would be https://jira.apache.org/jira/browse/ARROW-5809. Once ported to docker/docker-compose, it's trivial to activate github action for the same test (see https://github.com/apache/arrow/pull/5530). As I'm

Re: Travis CI delays

2019-09-27 Thread Krisztián Szűcs
On Fri, Sep 27, 2019 at 5:07 PM Andy Grove wrote: > Thanks Krisztian. That's very helpful. I will create a CI page on the wiki > and add this info. > > Does anyone have any objections to me trying out GitHub Actions for running > the Rust tests on PR builds? I could try this out on my own fork fi

Re: Travis CI delays

2019-09-27 Thread Andy Grove
Thanks Krisztian. That's very helpful. I will create a CI page on the wiki and add this info. Does anyone have any objections to me trying out GitHub Actions for running the Rust tests on PR builds? I could try this out on my own fork first. On Fri, Sep 27, 2019 at 7:40 AM Krisztián Szűcs wrote:

Re: Travis CI delays

2019-09-27 Thread Krisztián Szűcs
On Fri, Sep 27, 2019 at 3:25 PM Andy Grove wrote: > I've been poking around on the Arrow website and wiki and I can't find > documentation relating to CI. Do we have any documentation on how things > work today or what the goals are? I don't think so. > For Rust builds it isn't immediately > ob

Re: Travis CI delays

2019-09-27 Thread Wes McKinney
There are a bunch of other mailing list discussions over the last few months I recommend digging up and reviewing. As one practical issue with the Ursabot builds, those are running on servers that are hosted at my physical home in Nashville. If there's a power outage https://ci.ursalabs.org and al

Re: Travis CI delays

2019-09-27 Thread Andy Grove
I've been poking around on the Arrow website and wiki and I can't find documentation relating to CI. Do we have any documentation on how things work today or what the goals are? For Rust builds it isn't immediately obvious why they are building on both Travis CI and Ursabot. On Fri, Sep 27, 2019

Re: Travis CI delays

2019-09-27 Thread Wes McKinney
I've been sounding the alarm bells about this for a while. We need to work to get ourselves off of Travis CI, but it is not going to be easy. On Thu, Sep 26, 2019 at 11:42 PM Micah Kornfield wrote: > > My understanding is the Travis CI queue is shared among all apache > projects, and there are fe

Re: Travis CI delays

2019-09-27 Thread Wes McKinney
FYI, using the kibble.dev link from INFRA-18533, it seems that this September we're using about 15% of the ASF's total Travis CI capacity (60 concurrent workers I think) https://imgur.com/a/oOrbPsj The highest is Apache Druid (incubating) at 18%, so we are #2. Suffice to say the ASF's Travis coul

Re: Travis CI delays

2019-09-26 Thread Micah Kornfield
My understanding is the Travis CI queue is shared among all apache projects, and there are few including Arrow that make heavy use of the resources. Hence, a lot of time waiting for jobs to start. I think there are some open JIRAs to finish dockerization of builds, I don't know the current status

Travis CI delays

2019-09-26 Thread Andy Grove
I know this has been discussed in the past, and I apologize for not paying attention at the time (and searching for arrow + travis in email isn't very effective) but why does it take so long for our Travis CI builds and are there open JIRA issues related to this? Thanks, Andy.