hi Krisztian, Before talking about any code donations or where to run builds, I think we first need to discuss the worrisome situation where we have in some cases 3 (or more) CI configurations for different components in the project.
Just taking into account out C++ build, we have: * A config for Travis CI * Multiple configurations in Dockerfiles under cpp/ * A brand new (?) configuration in this third party ursa-labs/ursabot repository I note for example that the "AMD64 Conda C++" Buildbot build is failing while Travis CI is succeeding https://ci.ursalabs.org/#builders/66/builds/3196 Starting from first principles, at least for Linux-based builds, what I would like to see is: * A single build configuration (which can be driven by yaml-based configuration files and environment variables), rather than 3 like we have now. This build configuration should be decoupled from any CI platform, including Travis CI and Buildbot * A local tool to run any Linux-based builds locally using Docker at the command line, so that CI behavior can be exactly reproduced locally Does that seem achievable? Thanks, Wes On Mon, Jul 29, 2019 at 6:22 PM Krisztián Szűcs <szucs.kriszt...@gmail.com> wrote: > > Hi All, > > Ursabot works pretty well so far, and the CI feedback times have become > even better* after enabling the docker volume caches, the development > and maintenance of it is still not available for the whole Arrow community. > > While it wasn't straightforward I've managed to separate to source code > required to configure the Arrow builders into a separate directory, which > eventually can be donated to Arrow. > The README is under construction, but the code is available here [1]. > > Until this codebase is not governed by the Arrow community, > decommissioning slow travis builds is not possible, so the overall CI times > required to merge a PR will remain high. > > Regards, Krisztian > > * C++ builder times have dropped from ~6-7 minutes to ~3-4 minutes > * Python builder times have dropped from ~7-8 minutes to ~3-5 minutes > * ARM C++ builder time have dropped from ~19-20 minutes to ~9-12 minutes > > [1]: > https://github.com/ursa-labs/ursabot/tree/a46c6aa7b714346b3e4bb7921decb4d4d2f5ed70/projects/arrow
