[
https://issues.apache.org/jira/browse/SPARK-37865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Rosen updated SPARK-37865:
-------------------------------
Labels: correctness (was: )
> Spark should not dedup the groupingExpressions when the first child of Union
> has duplicate columns
> --------------------------------------------------------------------------------------------------
>
> Key: SPARK-37865
> URL: https://issues.apache.org/jira/browse/SPARK-37865
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.3.0
> Reporter: Chao Gao
> Assignee: Karen Feng
> Priority: Major
> Labels: correctness
> Fix For: 3.1.3, 3.0.4, 3.3.0, 3.2.2
>
>
> When the first child of Union has duplicate columns like select a, a from t1
> union select a, b from t2, spark only use the first column to aggregate the
> results, which would make the results incorrect, and this behavior is
> inconsistent with other engines like PostgreSQL, MySQL. We could alias the
> attribute of the first child of union to resolve this, or you could argue
> that this is the feature of Spark SQL.
> sample query:
> select
> a,
> a
> from values (1, 1), (1, 2) as t1(a, b)
> UNION
> SELECT
> a,
> b
> from values (1, 1), (1, 2) as t2(a, b)
> result is
> (1,1)
> result from PostgreSQL and MySQL
> (1,1)
> (1,2)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
