On Fri, Sep 6, 2019 at 7:56 PM Wes McKinney <wesmck...@gmail.com> wrote:
> On Fri, Sep 6, 2019 at 3:18 AM Krisztián Szűcs > <szucs.kriszt...@gmail.com> wrote: > > > > Hey Wes, > > > > On Fri, Sep 6, 2019 at 12:23 AM Wes McKinney <wesmck...@gmail.com> > wrote: > > > > > hi Krisztian, > > > > > > Anyone who's developing in the project can see that the Buildbot setup > > > is working well (at least for Linux builds) and giving much more > > > timely feedback, which has been very helpful. > > > > > > I'm concerned about the "ursabot" approach for a few reasons: > > > > > > * If we are to centralize our tooling for Arrow CI builds, why can we > > > not have the build tool itself under Arrow governance? > > > > > See below. > > > > > * The current "ursabot" tool has GPL dependencies. Can these be > > > factored out into plugins so that the tool itself is ASF-compatible? > > > > Ursabot is actually a buildbot plugin, however it contains some vendored > > code from buildbot. If we can push those fixes upstream to buildbot, then > > ursabot can be ASF compatible, thus may be maintained within arrow. > > > > > * This is a bit nitpicky but the name "ursabot" bears the name mark of > > > an organization that funds developers in this project. I'm concerned > > > about this, as I would about a tool named "clouderabot", "dremiobot", > > > "databricksbot", "googlebot", "ibmbot" or anything like that. It's > > > different from using a tool developed by an unaffiliated third party > > > > > Ursa-labs is concentrated to the development of Arrow, so I think it is a > > bit different than your examples. > > We can rename it if you want or resolve the licensing of ursabot (push > > all of the vendored code to buildbot), then donate it to Arrow. > > > > You're suggesting that one organization that contributes to Apache > Arrow deserves preferential treatment over others. This is not > consistent with Apache project independence > It wasn't my intention to suggest that. We just need to have a repository for the `arrow buildbot plugin` and GitHub user to interact with. We can move this repository to any GitHub organization or user, and we can pick arbitrary name for both the repository and the github machine account. > > https://community.apache.org/projectIndependence.html > > "Apache projects must be managed independently, and PMCs must ensure > that they are acting in the best interests of the project as a whole. > Note that it is similarly important that the PMC clearly show this > independence within their project community. The perception of > existing and new participants within the community that the PMC is run > independently and without favoring any specific third parties over > others is important, to allow new contributors to feel comfortable > both joining the community and contributing their work. A community > that obviously favors one specific vendor in some exclusive way will > often discourage new contributors from competing vendors, which is an > issue for the long term health of the project." > > > > > > > In any case, I think putting the build configurations for the current > > > Ursa Labs-managed build cluster in the Apache Arrow repository is a > > > good idea, but there are likely a number of issues that we need to > > > address to be able to contemplate having a hard dependency between the > > > CI that we depend on to merge patches and this tool. > > > > > > - Wes > > > > > > On Thu, Sep 5, 2019 at 8:17 AM Antoine Pitrou <anto...@python.org> > wrote: > > > > > > > > > > > > Le 05/09/2019 à 15:04, Krisztián Szűcs a écrit : > > > > >> > > > > >> If going with buildbot, this means that the various build steps > need > > > to > > > > >> be generic like in Travis-CI (e.g. "install", "setup", > "before-test", > > > > >> "test", "after-test"...) and their contents expressed outside of > the > > > > >> buildmaster configuration per se. > > > > >> > > > > > This is partially resolved with the Builder abstraction, see an > example > > > > > here [1]. We just need to add and reload these Builder > configurations > > > > > dynamically on certain events, like when someone changes a builder > > > > > from a PR. > > > > > > > > This is inside the buildmaster process, right? I don't understand > how > > > > you plan to change those dynamically without affecting all concurrent > > > > builds. > > > > > > > > Regards > > > > > > > > Antoine. > > > >
