westonpace commented on pull request #11876: URL: https://github.com/apache/arrow/pull/11876#issuecomment-994137086
> Regarding the benchmark being too slow: Making the number of selectivity test cases smaller and removing a lot of the slow utf8 runs, it now runs in 16.64 mins on my machine, which is more reasonable? It would certainly be an outlier. Right now a full run of 52 benchmarks (including compilation) takes ~ 100 minutes. So we are averaging ~2 minutes per benchmark (some are quite a bit faster and some are quite a bit slower). I don't have the distribution for all of them. Benchmarking tools like conbench run the full suite of C++ benchmarks against every Arrow commit. So at the very least I think there would need to be a compelling case why we can't get away with something similar here (again, it may be we want a reduced "for-automation" set which is the default and a more complete "for-investigation" set) CC @jonkeane @pitrou for a second opinion -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
