omar created SPARK-37560: ---------------------------- Summary: equality comparison for map type field Key: SPARK-37560 URL: https://issues.apache.org/jira/browse/SPARK-37560 Project: Spark Issue Type: Improvement Components: Spark Core Affects Versions: 2.4.0 Reporter: omar
hi where I try to execute this kind of qury {code:java} SELECT * FROM myTable where myMapField is null {code} I get this king of error {code:java} org.apache.spark.sql.AnalysisException: Cannot have map type columns in DataFrame which calls set operations(intersect, except, etc.), but the type of column is map ... {code} According to this link [https://github.com/apache/spark/blob/branch-2.4/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala#L406] would it be possible to assign a default comparaison function for Map type ? i'm working on spark 2.4, and it seams that this issue happens with version 3.2 too (same comment in 3.2 branch) -- This message was sent by Atlassian Jira (v8.20.1#820001) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org