omar created SPARK-37560:
----------------------------
Summary: equality comparison for map type field
Key: SPARK-37560
URL: https://issues.apache.org/jira/browse/SPARK-37560
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 2.4.0
Reporter: omar
hi
where I try to execute this kind of qury
{code:java}
SELECT * FROM myTable where myMapField is null {code}
I get this king of error
{code:java}
org.apache.spark.sql.AnalysisException: Cannot have map type columns in
DataFrame which calls set operations(intersect, except, etc.), but the type of
column is map ... {code}
According to this link
[https://github.com/apache/spark/blob/branch-2.4/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala#L406]
would it be possible to assign a default comparaison function for Map type ?
i'm working on spark 2.4, and it seams that this issue happens with version 3.2
too (same comment in 3.2 branch)
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
