Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22514#discussion_r227653464
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveParquetSuite.scala ---
@@ -92,4 +92,18 @@ class HiveParquetSuite extends QueryTest with
ParquetTest with TestHiveSingleton
}
}
}
+
+ test("SPARK-25271: write empty map into hive parquet table") {
+ val testData =
hiveContext.getHiveFile("data/files/empty_map.dat").getCanonicalFile()
+ val sourceTable = "sourceTable"
+ val targetTable = "targetTable"
+ withTable(sourceTable, targetTable) {
+ sql(s"CREATE TABLE $sourceTable (i int,m map<int, string>) ROW
FORMAT DELIMITED FIELDS " +
+ "TERMINATED BY ',' COLLECTION ITEMS TERMINATED BY ':' MAP KEYS
TERMINATED BY '$'")
+ sql(s"LOAD DATA LOCAL INPATH '${testData.toURI}' INTO TABLE
$sourceTable")
--- End diff --
can we generate the input data with a temp view? e.g. create a dataframe
with literals and register temp view.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
