Thanks for this. I also don’t know of anyone else using conbench currently, but I am happy to hear that you are interested in (having your agents) work on it.
-Jon > On Jun 4, 2026, at 16:45, Rok Mihevc <[email protected]> wrote: > > Will do! Looking forward to hearing what you're looking to achieve. > > Perhaps we can use the attention here (if we got any) to discuss what Arrow > needs/wants in terms of benchmarking functionality. > > > On Thu, Jun 4, 2026 at 9:55 PM Wes McKinney <[email protected]> wrote: > >> Okay sounds good! I’ll start a v2 branch and do some work on it. Can you >> contact me offline and let me know how I can get a copy of the prod >> database to help with development and test migrations? >> >> Thanks >> Wes >> >> On Thu, Jun 4, 2026 at 12:57 Rok Mihevc <[email protected]> wrote: >> >>> As per my discussion with the Velox team in March, they weren't using >>> Conbench at the time and had no plans to migrate back. >>> >>> I never reported on the Conbench/Voltron migration to the mailing list, >> so >>> this is a good opportunity to do so: >>> - In November, it became clear Voltron Data would shut down its CI and >>> benchmarking infrastructure >>> - Using the donated AWS credits [1] and with the help of Mike Wendt I >> moved >>> conbench infra to the new AWS account and backfilled benchmarks to cover >>> the missing commits >>> - Raul and I moved CUDA runners to the new AWS account >>> - Moved doc-preview, Parquet benchmarking/testing datasets, the nightly >>> dashboard, etc. to S3 with help from Nic, Jacob, and Raul >>> - Conbench gradually stabilized, but benchmarks occasionally fail due to >>> dependency issues. We don't really track these, but Nic and I typically >> fix >>> them. >>> - The database design still makes the web views slow, and we discussed >>> about dropping some history to speed them up >>> - AWS benchmark runner-minutes consume nearly the entire donated AWS >> quota. >>> Discussion on how to optimize this and what exactly we need from >> benchmarks >>> as a project occasionally comes up on Zulip, and I expect that once it >>> crystallizes we'll bring it to the ML as well. >>> - Things are now stable enough that we can discuss redesigning toward a >>> more stable and sustainable setup >>> >>> Best, >>> Rok >>> >>> [1] https://lists.apache.org/thread/q33oofy2v3zpg9s9l8o0w68rmjr3ocsv >>> >>> On Thu, Jun 4, 2026 at 5:10 PM Jacob Wujciak <[email protected]> >>> wrote: >>> >>>> +1 improving conbench and the orchestration would free up aws credits >> and >>>> more importantly the time taken to fix outrages! >>>> >>>> Velox used to use conbench but through a VD provided instance. I >> haven't >>>> seen any movement to set up an independent instance. >>>> >>>> >>>> Rok Mihevc <[email protected]> schrieb am Do., 4. Juni 2026, 16:38: >>>> >>>>> Hi Wes, >>>>> >>>>> Conbench is now at conbench.arrow-dev.org. I forked all repos to >>>>> arctosalliance github org in case VD deleted them on the way out. >>>>> Historical RDS db is preserved in the AWS account we migrated Arrow's >>> CI >>>>> infra to. It's supported by donated AWS credits. >>>>> For orchestration buildkite.com/apache-arrow is still used and I >> even >>>> got >>>>> the old M1 mac mini from Mike. It's now in my rack running benchmarks >>>>> again. >>>>> >>>>> That said current design of the orchestration layer, api server and >> db >>> is >>>>> somewhat costly and brittle. Downtime and adhoc fixes are common and >> I >>>> was >>>>> thinking about refactoring it somewhat to make it more maintainable. >>>>> >>>>> 1) If things become more maintainable and long term stable that'd be >>>> great. >>>>> I'm happy to collaborate on this. >>>>> 2) I'm only aware of arrow and arrow-go currently using it. >>>>> >>>>> Best, >>>>> Rok >>>>> >>>>> On Thu, Jun 4, 2026 at 4:20 PM Wes McKinney <[email protected]> >>> wrote: >>>>> >>>>>> hi all, >>>>>> >>>>>> I saw that conbench.ursa.dev has been down and I had a need to set >>> up >>>>>> some continuous project benchmarks, and was interested in doing >>>>>> development on Conbench (well, having my agents do development on >>>>>> Conbench), and was interested in the following: >>>>>> >>>>>> 1) is there interest in migrating the historical Arrow conbench >> data >>>>>> to a new server, has that been preserved somewhere? I'll probably >>>>>> rewrite the conbench backend in Go and give it a client CLI for >>>>>> submitting new data or querying old data. >>>>>> >>>>>> 2) are there other users of conbench (conbench/conbench) that >> anyone >>>>>> is aware of? I'd be done doing in-situ development in that >> repository >>>>>> or setting up a conbench-v2 project. >>>>>> >>>>>> No particular urgency but if anyone has opinions let me know! >>>>>> >>>>>> thanks, >>>>>> Wes >>>>>> >>>>> >>>> >>> >>
