[
https://issues.apache.org/jira/browse/SPARK-44082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17737265#comment-17737265
]
Ignite TC Bot commented on SPARK-44082:
---------------------------------------
User 'amaliujia' has created a pull request for this issue:
https://github.com/apache/spark/pull/41633
> Generate operator does not update reference set properly
> --------------------------------------------------------
>
> Key: SPARK-44082
> URL: https://issues.apache.org/jira/browse/SPARK-44082
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.5.0
> Reporter: Rui Wang
> Assignee: Rui Wang
> Priority: Major
>
> Before
> ```
> == Optimized Logical Plan ==
> Project [col1#2, col2#19]
> +- Generate replicaterows(sum#17L, col1#2, col2#3), [2], false, [col1#2,
> col2#3]
> +- Filter (isnotnull(sum#17L) AND (sum#17L > 0))
> +- Aggregate [col1#2, col2#19], [col1#2, col2#19, sum(vcol#14L) AS
> sum#17L]
> +- Union false, false
> :- Aggregate [col1#2], [1 AS vcol#14L, col1#2, first(col2#3,
> false) AS col2#19]
> : +- LogicalRDD [col1#2, col2#3], false
> +- Project [-1 AS vcol#15L, col1#8, col2#9]
> +- LogicalRDD [col1#8, col2#9], false
> ```
> Couldn't find col2#3 in [col1#2,col2#19,sum#17L]
> after
> ```
> == Optimized Logical Plan ==
> Project [col1#2, col2#19]
> +- Generate replicaterows(sum#17L, col1#2, col2#19), [2], false, [col1#2,
> col2#19]
> +- Filter (isnotnull(sum#17L) AND (sum#17L > 0))
> +- Aggregate [col1#2, col2#19], [col1#2, col2#19, sum(vcol#14L) AS
> sum#17L]
> +- Union false, false
> :- Aggregate [col1#2], [1 AS vcol#14L, col1#2, first(col2#3,
> false) AS col2#19]
> : +- LogicalRDD [col1#2, col2#3], false
> +- Project [-1 AS vcol#15L, col1#8, col2#9]
> +- LogicalRDD [col1#8, col2#9], false
> ```
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
