We'd like to have the conbench project itself exist as an independent open source project so it can be easily used in other projects. If it makes sense to donate to the ASF at some point we are open to the discussion! In the meantime, we welcome pull requests.
Regarding the Arrow benchmarks, in principle having these in an Arrow repository (along with various other benchmarking infrastructure) would make sense. One of the reasons for a separate repository is to not bind the benchmarks to a particular git hash of the Arrow codebase (so data can be backfilled). We have the same issue with collecting C++ benchmark data from old versions of the project with newer benchmarks (at least when the API has not changed, so things would still compile with the old version of the codebase) On Mon, May 10, 2021 at 3:31 PM Sutou Kouhei <k...@clear-code.com> wrote: > > Great! > > Do you have a plan to donate this and Arrow R/Python > benchmarks to Apache Arrow project? Should we keep them > under https://github.com/ursacomputing/ ? > > Thanks, > -- > kou > > In <caezaprbt2i0iv+xabaxfwqhjyj048s16peunofbdkrhakyu...@mail.gmail.com> > "Announcing Conbench + Arrow" on Mon, 10 May 2021 12:11:45 -0600, > Diana Clarke <diana.joan.cla...@gmail.com> wrote: > > > Hi folks: > > > > Last week we officially announced a new benchmarking tool called > > Conbench with an Arrow integration. > > > > https://ursalabs.org/blog/announcing-conbench/ > > https://twitter.com/wesmckinn/status/1390324198623547392 > > > > Conbench is a language independent, continuous benchmarking (CB) > > framework, built specifically with the needs of a cross-language, > > platform-independent, high-performance project like Arrow in mind. > > > > https://github.com/ursacomputing/conbench > > > > On each merge to the main Arrow branch, over 2000 C++, R, and Python > > benchmarks are run, and the results are posted to our publicly > > available Conbench server (click the following link to see all your > > pretty faces and avatars). > > > > https://conbench.ursa.dev/ > > > > You can also benchmark your Arrow pull requests with the GitHub > > comment: "@ursabot please benchmark". More information about how to > > benchmark your pull requests can be found in the blog post linked to > > above. > > > > A *huge* thanks to Elena Henderson for all the orchestration and > > behind the scenes system administration & services she built – I > > couldn't have asked for a better partner to collaborate on this > > project with. > > > > Thanks also to Jonathan Keane & Neal Richardson for the R benchmarks, > > and to Antoine Pitrou, Weston Pace, David Li, Krisztián Szucs, Wes > > McKinney, and many others for lending their domain expertise and > > support along the way. > > > > I'll circle back in a month or so with an update on other Conbench > > projects currently in flight. Sneak preview: better statistical > > analysis courtesy of Jonathan Keane, and Java benchmarks courtesy of > > Kazuaki Ishizaki. In the meantime, I wish you all speedy access to > > vaccines. > > > > --diana
