Deepa Vasanthkumar created SPARK-38087:
------------------------------------------
Summary: select doesnt validate if the column already exists
Key: SPARK-38087
URL: https://issues.apache.org/jira/browse/SPARK-38087
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 3.2.1
Environment: Version{{{}v3.2.1{}}}
{{}}
{{{}{}}}Master{{{}local[*]{}}}
{{(Reproducible in any environment)}}
Reporter: Deepa Vasanthkumar
Fix For: 3.3
Select doesnt validate whether the alias column is already present in the
dataframe.
However drop will not let you drop the said column.
Scenario to reproduce :
df2 = df1.select("*", (df1.firstname).alias("firstname")) ---> this will add
same column
df2.show()
df2.drop(df2.firstname) --> this will give AnalysisException: Reference
'firstname' is ambiguous, could be: firstname, firstname.
Is this expected behavior .
!image-2022-02-02-06-28-23-543.png!
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
