Re: New CI system: Ursabot

2019-07-29 Thread Krisztián Szűcs
It was resolved with https://github.com/ursa-labs/ursabot/pull/118 See the builders here https://ci.ursalabs.org/#/builders?tags=%2Bcuda On Tue, Jun 25, 2019 at 9:54 PM Wes McKinney wrote: > Krisz -- can the CUDA issue be tracked somewhere so we remember to do > it (and set ARROW_CUDA=on)? > > O

Re: New CI system: Ursabot

2019-07-11 Thread Krisztián Szűcs
Once the latter one is set up, I can also grant access for contributors outside of apache. BTW I think you can safely ignore the rust failure because it uses the nightly toolchain. > > Eric > > -Original Message- > From: Krisztián Szűcs > Sent: Friday, June 14, 2019

RE: New CI system: Ursabot

2019-07-11 Thread Eric Erhardt
bot legs. Eric -Original Message- From: Krisztián Szűcs Sent: Friday, June 14, 2019 9:48 AM To: dev@arrow.apache.org Subject: New CI system: Ursabot Hello All, We're developing a buildbot application to utilize Ursa Labs’ physical machines called Ursabot. Buildbot [1] is used by major o

Re: New CI system: Ursabot

2019-06-25 Thread Wes McKinney
Krisz -- can the CUDA issue be tracked somewhere so we remember to do it (and set ARROW_CUDA=on)? On Mon, Jun 24, 2019 at 4:42 AM Krisztián Szűcs wrote: > > We already have a CUDA builder in ursabot [1], just need to enable > --runtime=nvidia for the docker worker. > > [1]: > https://github.com/u

Re: New CI system: Ursabot

2019-06-24 Thread Krisztián Szűcs
We already have a CUDA builder in ursabot [1], just need to enable --runtime=nvidia for the docker worker. [1]: https://github.com/ursa-labs/ursabot/blob/master/ursabot/builders.py#L445 On Fri, Jun 21, 2019 at 9:58 PM Keith Kraus wrote: > There's nvidia-docker (https://github.com/NVIDIA/nvidia-

Re: New CI system: Ursabot

2019-06-21 Thread Keith Kraus
There's nvidia-docker (https://github.com/NVIDIA/nvidia-docker) which handles passing through the GPU devices and necessary driver modules into a docker container. CUDA doesn't get mapped in as it's userspace so you'll need to either use an image with CUDA baked in (i.e. https://hub.docker.com/

Re: New CI system: Ursabot

2019-06-21 Thread Antoine Pitrou
Is it possible to test CUDA under a Docker container? I feel like I'm the only person who routinely tests CUDA on my home machine :-) And of course I only do that on Linux... Regards Antoine. On Fri, 21 Jun 2019 12:23:10 -0500 Wes McKinney wrote: > hi folks, > > I would suggest the followi

Re: New CI system: Ursabot

2019-06-21 Thread Francois Saint-Jacques
Yes please. Locally reproducible failed builds/tests should be a top priority. I would also propose that we cache docker images on a nightly basis and export them via quay or other channel. I spent the last days looking at docker build with wall of apt and conda lines. François On Fri, Jun 21,

Re: New CI system: Ursabot

2019-06-21 Thread Wes McKinney
hi folks, I would suggest the following development approach to help with increasing our CI capacity: 1. For all Linux builds, refactor Travis CI jobs to be Docker-based and not depend on Travis-CI-specific state or environment variables 2. Add such jobs to Ursabot. If there is satisfaction with

Re: New CI system: Ursabot

2019-06-17 Thread Krisztián Szűcs
That's right, OWNER, MEMBER and CONTRIBUTOR roles are allowed: CONTRIBUTOR Author has previously committed to the repository. MEMBER Author is a member of the organization that owns the repository. OWNER Author is the owner of the repository. See https://developer.github.com/v4/enum/commentautho

Re: New CI system: Ursabot

2019-06-17 Thread Wes McKinney
On Mon, Jun 17, 2019 at 7:25 AM Krisztián Szűcs wrote: > > On Sun, Jun 16, 2019 at 6:17 AM Micah Kornfield > wrote: > > > Hi Krisztian, > > This is really cool, thank you for doing this. Two questions: > > 1. How reliable is the build setup? Is it reliable enough at this point to > > be consid

Re: New CI system: Ursabot

2019-06-17 Thread Krisztián Szűcs
On Sun, Jun 16, 2019 at 6:17 AM Micah Kornfield wrote: > Hi Krisztian, > This is really cool, thank you for doing this. Two questions: > 1. How reliable is the build setup? Is it reliable enough at this point to > be considered a merge blocker if a build fails? > IMO yes. > 2. What is the p

Re: New CI system: Ursabot

2019-06-16 Thread Uwe L. Korn
On Fri, Jun 14, 2019, at 11:23 PM, Krisztián Szűcs wrote: > On Fri, Jun 14, 2019 at 9:04 PM Wes McKinney wrote: > > > hi Krisz, > > > > Thanks for working on this! It already helped me fix a Python 2.7-only > > bug yesterday https://github.com/apache/arrow/pull/4553 > > > > I have a bunch of q

Re: New CI system: Ursabot

2019-06-15 Thread Micah Kornfield
Hi Krisztian, This is really cool, thank you for doing this. Two questions: 1. How reliable is the build setup? Is it reliable enough at this point to be considered a merge blocker if a build fails? 2. What is the permission model for triggering runs? Is it open to anybody on github? Only Urs

Re: New CI system: Ursabot

2019-06-14 Thread Antoine Pitrou
Le 14/06/2019 à 23:22, Krisztián Szűcs a écrit : >> >> * Do machines have to be co-located on the same physical network as >> the master, or can they reside in other locations? >> > It is preferable to have a master in the same network where the workers are, > because the build steps are rpc call

Re: New CI system: Ursabot

2019-06-14 Thread Krisztián Szűcs
On Fri, Jun 14, 2019 at 9:04 PM Wes McKinney wrote: > hi Krisz, > > Thanks for working on this! It already helped me fix a Python 2.7-only > bug yesterday https://github.com/apache/arrow/pull/4553 > > I have a bunch of questions: > > * What is the license of the ursabot codebase? Seems like it co

Re: New CI system: Ursabot

2019-06-14 Thread Wes McKinney
hi Krisz, Thanks for working on this! It already helped me fix a Python 2.7-only bug yesterday https://github.com/apache/arrow/pull/4553 I have a bunch of questions: * What is the license of the ursabot codebase? Seems like it could be GPL if Buildbot itself is [2] and you have reused any Buildb

New CI system: Ursabot

2019-06-14 Thread Krisztián Szűcs
Hello All, We're developing a buildbot application to utilize Ursa Labs’ physical machines called Ursabot. Buildbot [1] is used by major open source projects, like CPython and WebKit [2]. The source code is hosted at [3], the web interface is accessible at [4]. The repository contains a short gui