[ https://issues.apache.org/jira/browse/SPARK-37560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon resolved SPARK-37560. ---------------------------------- Resolution: Duplicate > equality comparison for map type field > -------------------------------------- > > Key: SPARK-37560 > URL: https://issues.apache.org/jira/browse/SPARK-37560 > Project: Spark > Issue Type: Improvement > Components: Spark Core > Affects Versions: 2.4.0 > Reporter: omar > Priority: Major > > hi > where I try to execute this kind of qury > > {code:java} > SELECT * FROM myTable where myMapField is null {code} > I get this king of error > > {code:java} > org.apache.spark.sql.AnalysisException: Cannot have map type columns in > DataFrame which calls set operations(intersect, except, etc.), but the type > of column is map ... {code} > According to this link > [https://github.com/apache/spark/blob/branch-2.4/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala#L406] > > > would it be possible to assign a default comparaison function for Map type ? > i'm working on spark 2.4, and it seams that this issue happens with version > 3.2 too (same comment in 3.2 branch) > -- This message was sent by Atlassian Jira (v8.20.1#820001) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org