maropu commented on a change in pull request #27926: [SPARK-31166][SQL] UNION map<null, null> and other maps should not fail URL: https://github.com/apache/spark/pull/27926#discussion_r393389988
########## File path: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala ########## @@ -3487,6 +3487,12 @@ class SQLQuerySuite extends QueryTest with SharedSparkSession with AdaptiveSpark } } } + + test("SPARK-31166: UNION map<null, null> and other maps should not fail") { + checkAnswer( + sql("(SELECT map()) UNION ALL (SELECT map(1, 2))"), + Seq(Row(Map[Int, Int]()), Row(Map(1 -> 2)))) + } Review comment: Nice catch! It seems #27542 broke some other queries merging `map<null,null>` and `map<some type, some type>`, e.g., `map_concat`; ``` // the current master scala> sql("select map_concat(map(), map(1, 2))").show() org.apache.spark.sql.AnalysisException: cannot resolve 'map_concat(map(), map(1, 2))' due to data type mismatch: input to function map_concat should all be the same type, but it's [map<null,null>, map<int,int>]; line 1 pos 7; 'Project [unresolvedalias(map_concat(map(), map(1, 2)), None)] +- OneRowRelation at org.apache.spark.sql.catalyst.analysis.package$AnalysisErrorAt.failAnalysis(package.scala:42) ``` This PR can fix this query, too. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org