wuyi created SPARK-34319:
----------------------------
Summary: Self-join after cogroup applyInPandas fails due to
unresolved conflicting attributes
Key: SPARK-34319
URL: https://issues.apache.org/jira/browse/SPARK-34319
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 3.0.1, 3.0.0, 3.1.0, 3.2.0
Reporter: wuyi
{code:java}
df = spark.createDataFrame([(1, 1)], ("column", "value"))row =
df.groupby("ColUmn").cogroup(
df.groupby("COLUMN")
).applyInPandas(lambda r, l: r + l, "column long, value long")
row.join(row).show()
{code}
{code:java}
Conflicting attributes: column#163321L,value#163322L
;;
’Join Inner
:- FlatMapCoGroupsInPandas [ColUmn#163312L], [COLUMN#163312L],
<lambda>(column#163312L, value#163313L, column#163312L, value#163313L),
[column#163321L, value#163322L]
: :- Project [ColUmn#163312L, column#163312L, value#163313L]
: : +- LogicalRDD [column#163312L, value#163313L], false
: +- Project [COLUMN#163312L, column#163312L, value#163313L]
: +- LogicalRDD [column#163312L, value#163313L], false
+- FlatMapCoGroupsInPandas [ColUmn#163312L], [COLUMN#163312L],
<lambda>(column#163312L, value#163313L, column#163312L, value#163313L),
[column#163321L, value#163322L]
:- Project [ColUmn#163312L, column#163312L, value#163313L]
: +- LogicalRDD [column#163312L, value#163313L], false
+- Project [COLUMN#163312L, column#163312L, value#163313L]
+- LogicalRDD [column#163312L, value#163313L], false
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
