Re: New CI system: Ursabot

2019-07-29 Thread Krisztián Szűcs
It was resolved with https://github.com/ursa-labs/ursabot/pull/118 See the builders here https://ci.ursalabs.org/#/builders?tags=%2Bcuda On Tue, Jun 25, 2019 at 9:54 PM Wes McKinney wrote: > Krisz -- can the CUDA issue be tracked somewhere so we remember to do > it (and set ARROW_CUDA=on)? > > O

Re: New CI system: Ursabot

2019-07-11 Thread Krisztián Szűcs
Hi Eric! On Thu, Jul 11, 2019 at 3:34 PM Eric Erhardt wrote: > My apologies if this is already covered in the docs, but I couldn't find > it. > > How do I re-run a single leg in the Ursabot tests? The 'AMD64 Debian 9 > Rust 1.35' failed on my PR, and I wanted to try re-running just that leg, > b

RE: New CI system: Ursabot

2019-07-11 Thread Eric Erhardt
My apologies if this is already covered in the docs, but I couldn't find it. How do I re-run a single leg in the Ursabot tests? The 'AMD64 Debian 9 Rust 1.35' failed on my PR, and I wanted to try re-running just that leg, but the only option I found was to re-run all Ursabot legs. Eric -Or

Re: New CI system: Ursabot

2019-06-25 Thread Wes McKinney
Krisz -- can the CUDA issue be tracked somewhere so we remember to do it (and set ARROW_CUDA=on)? On Mon, Jun 24, 2019 at 4:42 AM Krisztián Szűcs wrote: > > We already have a CUDA builder in ursabot [1], just need to enable > --runtime=nvidia for the docker worker. > > [1]: > https://github.com/u

Re: New CI system: Ursabot

2019-06-24 Thread Krisztián Szűcs
We already have a CUDA builder in ursabot [1], just need to enable --runtime=nvidia for the docker worker. [1]: https://github.com/ursa-labs/ursabot/blob/master/ursabot/builders.py#L445 On Fri, Jun 21, 2019 at 9:58 PM Keith Kraus wrote: > There's nvidia-docker (https://github.com/NVIDIA/nvidia-

Re: New CI system: Ursabot

2019-06-21 Thread Keith Kraus
There's nvidia-docker (https://github.com/NVIDIA/nvidia-docker) which handles passing through the GPU devices and necessary driver modules into a docker container. CUDA doesn't get mapped in as it's userspace so you'll need to either use an image with CUDA baked in (i.e. https://hub.docker.com/

Re: New CI system: Ursabot

2019-06-21 Thread Antoine Pitrou
Is it possible to test CUDA under a Docker container? I feel like I'm the only person who routinely tests CUDA on my home machine :-) And of course I only do that on Linux... Regards Antoine. On Fri, 21 Jun 2019 12:23:10 -0500 Wes McKinney wrote: > hi folks, > > I would suggest the followi

Re: New CI system: Ursabot

2019-06-21 Thread Francois Saint-Jacques
Yes please. Locally reproducible failed builds/tests should be a top priority. I would also propose that we cache docker images on a nightly basis and export them via quay or other channel. I spent the last days looking at docker build with wall of apt and conda lines. François On Fri, Jun 21,

Re: New CI system: Ursabot

2019-06-21 Thread Wes McKinney
hi folks, I would suggest the following development approach to help with increasing our CI capacity: 1. For all Linux builds, refactor Travis CI jobs to be Docker-based and not depend on Travis-CI-specific state or environment variables 2. Add such jobs to Ursabot. If there is satisfaction with

Re: New CI system: Ursabot

2019-06-17 Thread Krisztián Szűcs
That's right, OWNER, MEMBER and CONTRIBUTOR roles are allowed: CONTRIBUTOR Author has previously committed to the repository. MEMBER Author is a member of the organization that owns the repository. OWNER Author is the owner of the repository. See https://developer.github.com/v4/enum/commentautho

Re: New CI system: Ursabot

2019-06-17 Thread Wes McKinney
On Mon, Jun 17, 2019 at 7:25 AM Krisztián Szűcs wrote: > > On Sun, Jun 16, 2019 at 6:17 AM Micah Kornfield > wrote: > > > Hi Krisztian, > > This is really cool, thank you for doing this. Two questions: > > 1. How reliable is the build setup? Is it reliable enough at this point to > > be consid

Re: New CI system: Ursabot

2019-06-17 Thread Krisztián Szűcs
On Sun, Jun 16, 2019 at 6:17 AM Micah Kornfield wrote: > Hi Krisztian, > This is really cool, thank you for doing this. Two questions: > 1. How reliable is the build setup? Is it reliable enough at this point to > be considered a merge blocker if a build fails? > IMO yes. > 2. What is the p

Re: New CI system: Ursabot

2019-06-16 Thread Uwe L. Korn
On Fri, Jun 14, 2019, at 11:23 PM, Krisztián Szűcs wrote: > On Fri, Jun 14, 2019 at 9:04 PM Wes McKinney wrote: > > > hi Krisz, > > > > Thanks for working on this! It already helped me fix a Python 2.7-only > > bug yesterday https://github.com/apache/arrow/pull/4553 > > > > I have a bunch of q

Re: New CI system: Ursabot

2019-06-15 Thread Micah Kornfield
Hi Krisztian, This is really cool, thank you for doing this. Two questions: 1. How reliable is the build setup? Is it reliable enough at this point to be considered a merge blocker if a build fails? 2. What is the permission model for triggering runs? Is it open to anybody on github? Only Urs

Re: New CI system: Ursabot

2019-06-14 Thread Antoine Pitrou
Le 14/06/2019 à 23:22, Krisztián Szűcs a écrit : >> >> * Do machines have to be co-located on the same physical network as >> the master, or can they reside in other locations? >> > It is preferable to have a master in the same network where the workers are, > because the build steps are rpc call

Re: New CI system: Ursabot

2019-06-14 Thread Krisztián Szűcs
On Fri, Jun 14, 2019 at 9:04 PM Wes McKinney wrote: > hi Krisz, > > Thanks for working on this! It already helped me fix a Python 2.7-only > bug yesterday https://github.com/apache/arrow/pull/4553 > > I have a bunch of questions: > > * What is the license of the ursabot codebase? Seems like it co

Re: New CI system: Ursabot

2019-06-14 Thread Wes McKinney
hi Krisz, Thanks for working on this! It already helped me fix a Python 2.7-only bug yesterday https://github.com/apache/arrow/pull/4553 I have a bunch of questions: * What is the license of the ursabot codebase? Seems like it could be GPL if Buildbot itself is [2] and you have reused any Buildb