Will do! Looking forward to hearing what you're looking to achieve. Perhaps we can use the attention here (if we got any) to discuss what Arrow needs/wants in terms of benchmarking functionality.
On Thu, Jun 4, 2026 at 9:55 PM Wes McKinney <[email protected]> wrote: > Okay sounds good! I’ll start a v2 branch and do some work on it. Can you > contact me offline and let me know how I can get a copy of the prod > database to help with development and test migrations? > > Thanks > Wes > > On Thu, Jun 4, 2026 at 12:57 Rok Mihevc <[email protected]> wrote: > > > As per my discussion with the Velox team in March, they weren't using > > Conbench at the time and had no plans to migrate back. > > > > I never reported on the Conbench/Voltron migration to the mailing list, > so > > this is a good opportunity to do so: > > - In November, it became clear Voltron Data would shut down its CI and > > benchmarking infrastructure > > - Using the donated AWS credits [1] and with the help of Mike Wendt I > moved > > conbench infra to the new AWS account and backfilled benchmarks to cover > > the missing commits > > - Raul and I moved CUDA runners to the new AWS account > > - Moved doc-preview, Parquet benchmarking/testing datasets, the nightly > > dashboard, etc. to S3 with help from Nic, Jacob, and Raul > > - Conbench gradually stabilized, but benchmarks occasionally fail due to > > dependency issues. We don't really track these, but Nic and I typically > fix > > them. > > - The database design still makes the web views slow, and we discussed > > about dropping some history to speed them up > > - AWS benchmark runner-minutes consume nearly the entire donated AWS > quota. > > Discussion on how to optimize this and what exactly we need from > benchmarks > > as a project occasionally comes up on Zulip, and I expect that once it > > crystallizes we'll bring it to the ML as well. > > - Things are now stable enough that we can discuss redesigning toward a > > more stable and sustainable setup > > > > Best, > > Rok > > > > [1] https://lists.apache.org/thread/q33oofy2v3zpg9s9l8o0w68rmjr3ocsv > > > > On Thu, Jun 4, 2026 at 5:10 PM Jacob Wujciak <[email protected]> > > wrote: > > > > > +1 improving conbench and the orchestration would free up aws credits > and > > > more importantly the time taken to fix outrages! > > > > > > Velox used to use conbench but through a VD provided instance. I > haven't > > > seen any movement to set up an independent instance. > > > > > > > > > Rok Mihevc <[email protected]> schrieb am Do., 4. Juni 2026, 16:38: > > > > > > > Hi Wes, > > > > > > > > Conbench is now at conbench.arrow-dev.org. I forked all repos to > > > > arctosalliance github org in case VD deleted them on the way out. > > > > Historical RDS db is preserved in the AWS account we migrated Arrow's > > CI > > > > infra to. It's supported by donated AWS credits. > > > > For orchestration buildkite.com/apache-arrow is still used and I > even > > > got > > > > the old M1 mac mini from Mike. It's now in my rack running benchmarks > > > > again. > > > > > > > > That said current design of the orchestration layer, api server and > db > > is > > > > somewhat costly and brittle. Downtime and adhoc fixes are common and > I > > > was > > > > thinking about refactoring it somewhat to make it more maintainable. > > > > > > > > 1) If things become more maintainable and long term stable that'd be > > > great. > > > > I'm happy to collaborate on this. > > > > 2) I'm only aware of arrow and arrow-go currently using it. > > > > > > > > Best, > > > > Rok > > > > > > > > On Thu, Jun 4, 2026 at 4:20 PM Wes McKinney <[email protected]> > > wrote: > > > > > > > > > hi all, > > > > > > > > > > I saw that conbench.ursa.dev has been down and I had a need to set > > up > > > > > some continuous project benchmarks, and was interested in doing > > > > > development on Conbench (well, having my agents do development on > > > > > Conbench), and was interested in the following: > > > > > > > > > > 1) is there interest in migrating the historical Arrow conbench > data > > > > > to a new server, has that been preserved somewhere? I'll probably > > > > > rewrite the conbench backend in Go and give it a client CLI for > > > > > submitting new data or querying old data. > > > > > > > > > > 2) are there other users of conbench (conbench/conbench) that > anyone > > > > > is aware of? I'd be done doing in-situ development in that > repository > > > > > or setting up a conbench-v2 project. > > > > > > > > > > No particular urgency but if anyone has opinions let me know! > > > > > > > > > > thanks, > > > > > Wes > > > > > > > > > > > > > > >
