andygrove commented on issue #1519: URL: https://github.com/apache/datafusion-comet/issues/1519#issuecomment-2719078622
The first issue is that the Spark executor is configured to use 8GB, and with our default settings, Comet only gets 1.6 GB, which just isn't enough. If I change Spark executor memory to 4GB and specify Comet memory overhead of 4GB then the job runs with good performance (close to 2x). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
