[
https://issues.apache.org/jira/browse/SPARK-25450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Li updated SPARK-25450:
----------------------------
Description:
The problem was cause by the PushProjectThroughUnion rule, which, when creating
new Project for each child of Union, uses the same exprId for expressions of
the same position. This is wrong because, for each child of Union, the
expressions are all independent, and it can lead to a wrong result if other
rules like FoldablePropagation kicks in, taking two different expressions as
the same.
> PushProjectThroughUnion rule uses the same exprId for project expressions in
> each Union child, causing mistakes in constant propagation
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-25450
> URL: https://issues.apache.org/jira/browse/SPARK-25450
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.3.1
> Reporter: Maryann Xue
> Priority: Major
>
> The problem was cause by the PushProjectThroughUnion rule, which, when
> creating new Project for each child of Union, uses the same exprId for
> expressions of the same position. This is wrong because, for each child of
> Union, the expressions are all independent, and it can lead to a wrong result
> if other rules like FoldablePropagation kicks in, taking two different
> expressions as the same.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
