Cheng Su created SPARK-35179:
--------------------------------
Summary: Introduce hybrid join for sort merge join and shuffled
hash join in AQE
Key: SPARK-35179
URL: https://issues.apache.org/jira/browse/SPARK-35179
Project: Spark
Issue Type: Sub-task
Components: SQL
Affects Versions: 3.2.0
Reporter: Cheng Su
Per discussion in
[https://github.com/apache/spark/pull/32210#issuecomment-823503243] , we can
introduce some kind of {{HybridJoin}} operator in AQE, and we can choose to do
shuffled hash join vs sort merge join for each task independently, e.g. based
on partition size, task1 can do shuffled hash join, and task2 can do sort merge
join, etc.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
