Ke Jia created SPARK-29552:
------------------------------
Summary: Fix the flaky test failed in AdaptiveQueryExecSuite #
multiple joins
Key: SPARK-29552
URL: https://issues.apache.org/jira/browse/SPARK-29552
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 3.0.0
Reporter: Ke Jia
AQE will optimize the logical plan once there is query stage finished. So for
inner join, when two join side is all small to be the build side. The planner
of converting logical plan to physical plan will select the build side as
BuildRight if right side finished firstly or BuildLeft if left side finished
firstly. In some case, when BuildRight or BuildLeft may introduce additioanl
exchange to the parent node. The revert approach in OptimizeLocalShuffleReader
rule may be too conservative, which revert all the local shuffle reader when
introduce additional exchange not revert the local shuffle reader introduced
shuffle. It may be expense to only revert the local shuffle reader introduced
shuffle. The workaround is to apply the OptimizeLocalShuffleReader rule again
when creating new query stage to further optimize the sub tree shuffle reader
to local shuffle reader.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
