Asif created SPARK-45658:
----------------------------
Summary: Canonicalization of DynamicPruningSubquery is broken
Key: SPARK-45658
URL: https://issues.apache.org/jira/browse/SPARK-45658
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 3.5.0, 3.5.1
Reporter: Asif
The canonicalization of (buildKeys: Seq[Expression]) in the class
DynamicPruningSubquery is broken, as the buildKeys are canonicalized just by
calling
buildKeys.map(_.canonicalized)
The above would result in incorrect canonicalization as it would not be
normalizing the exprIds
The fix is to use the buildQuery : LogicalPlan's output to normalize the
buildKeys expression
as given below, using the standard approach.
buildKeys.map(QueryPlan.normalizeExpressions(_, buildQuery.output)),
Will be filing a PR and bug test for the same.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
