andygrove commented on PR #3549: URL: https://github.com/apache/datafusion-comet/pull/3549#issuecomment-3929803790
> > @Shekharrajak Thanks for looking at this, but I have some concerns about this approach. CI takes way too long already and adding this workflow seems like it would add even more overhead. I am not sure that GitHub runners will provide consistent performance for benchmarking, and we really need to be testing with large data sets for meaningful results. > > Committers already have the ability to trigger TPC-H benchmarks @ 100GB by commenting on PRs. This is quite new and experimental, so it hasn't been documented yet. These benchmarks run on dedicated hardware. > > So do we expect some kind of script that can help running benchmarks in k8s cluster in local? The benchmarks do run in k8s already, but using local mode rather than truly distributed. I am planning on making that change, and I also need to align this with the benchmark scripts currently in the repo, which I am working on refactoring in https://github.com/apache/datafusion-comet/pull/3538 and https://github.com/apache/datafusion-comet/pull/3539. I planned on supporting k8s as a future step after the docker-compose one gets merged. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
