[
https://issues.apache.org/jira/browse/SPARK-45658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Toth resolved SPARK-45658.
--------------------------------
Resolution: Duplicate
Let me close this as a duplicate because the issue has been fixed by
SPARK-56694.
Unfortunately while I was working on SPARK-56694
([https://github.com/apache/spark/pull/55644]), I didn't notice this previous
bug discocery and fix ([https://github.com/apache/spark/pull/49154]) offered by
[~ashahid7].
> Canonicalization of DynamicPruningSubquery is broken
> ----------------------------------------------------
>
> Key: SPARK-45658
> URL: https://issues.apache.org/jira/browse/SPARK-45658
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.5.0, 4.0.0
> Reporter: Asif
> Priority: Major
> Labels: pull-request-available
>
> The canonicalization of (buildKeys: Seq[Expression]) in the class
> DynamicPruningSubquery is broken, as the buildKeys are canonicalized just by
> calling
> buildKeys.map(_.canonicalized)
> The above would result in incorrect canonicalization as it would not be
> normalizing the exprIds relative to buildQuery output
> The fix is to use the buildQuery : LogicalPlan's output to normalize the
> buildKeys expression
> as given below, using the standard approach.
> buildKeys.map(QueryPlan.normalizeExpressions(_, buildQuery.output)),
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
