Re: [I] Unable to run TPC-H benchmarks using on-heap mode [datafusion-comet]
andygrove commented on issue #1519: URL: https://github.com/apache/datafusion-comet/issues/1519#issuecomment-2719078622 The first issue is that the Spark executor is configured to use 8GB, and with our default settings, Comet only gets 1.6 GB, which just isn't enough. If I change Spark executor memory to 4GB and specify Comet memory overhead of 4GB then the job runs with good performance (close to 2x). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
[I] Unable to run TPC-H benchmarks using on-heap mode [datafusion-comet]
andygrove opened a new issue, #1519: URL: https://github.com/apache/datafusion-comet/issues/1519 ### Describe the bug I use off-heap mode when running benchmarks locally, since this is the recommended approach for Comet. Today I tried running in on-heap mode and q4 seems to hang indefinitely. I will need to see if this is a regression since the last release and try and debug the issue farther. ### Steps to reproduce _No response_ ### Expected behavior _No response_ ### Additional context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
