Re: [I] Unable to run TPC-H benchmarks using on-heap mode and default settings [datafusion-comet]
andygrove closed issue #1519: Unable to run TPC-H benchmarks using on-heap mode and default settings URL: https://github.com/apache/datafusion-comet/issues/1519 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
Re: [I] Unable to run TPC-H benchmarks using on-heap mode and default settings [datafusion-comet]
wForget commented on issue #1519: URL: https://github.com/apache/datafusion-comet/issues/1519#issuecomment-2719621892 > The first issue is that the Spark executor is configured to use 8GB, and with our default settings, Comet only gets 1.6 GB, which just isn't enough. I had the same confusion before. https://github.com/apache/datafusion-comet/pull/1379#discussion_r1953731417 , maybe we can use executor off-heap memory (8g) as comet overhead memory. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
Re: [I] Unable to run TPC-H benchmarks using on-heap mode and default settings [datafusion-comet]
andygrove commented on issue #1519: URL: https://github.com/apache/datafusion-comet/issues/1519#issuecomment-2719205845 Spark requires at least 3GB to run the benchmarks. Comet also requires at least 3GB. In on-heap mode that means we require 6 GB total. In off-heap mode this is reduced to 5 GB. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
Re: [I] Unable to run TPC-H benchmarks using on-heap mode and default settings [datafusion-comet]
andygrove commented on issue #1519: URL: https://github.com/apache/datafusion-comet/issues/1519#issuecomment-2719338897 See https://github.com/apache/datafusion-comet/issues/1315 for more information. I plan on updating the tuning guide to provide guidance on specifying Comet memory allocation to close this issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
