Wei Xue created SPARK-39551:
-------------------------------
Summary: Add AQE invalid plan check
Key: SPARK-39551
URL: https://issues.apache.org/jira/browse/SPARK-39551
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 3.3.0
Reporter: Wei Xue
AQE logical optimization rules can lead to invalid physical plans as certain
physical plan nodes are not compatible with others. E.g.,
`BroadcastExchangeExec` can only work as a direct child of broadcast join nodes.
Logical optimizations, on the other hand, are not (and should not be) aware of
such restrictions. So a general solution here is to check for invalid plans and
throw exceptions, which can be caught by AQE replanning process. And if such an
exception is captured, AQE can void the current replanning result and keep
using the latest valid plan.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
