Dandandan commented on issue #16710: URL: https://github.com/apache/datafusion/issues/16710#issuecomment-3063067577
Besides profiling, I would like to suggest to research how the other engines are running the join and extract some high level learnings out of it: * is it using a hash table or open addressing (df doesn't have the latter) * are we using the correct/optimal join order? * is the hash-repartition taking considerable time (`RepartitionExec`, which involves 2 copies of the input) - what are the other engines doing? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
