This bug happens because the Glue table's SERDEPROPERTIES is missing two
important properties:
spark.sql.sources.schema.numParts
spark.sql.sources.schema.part.0
To solve the problem, I had to add those two properties via the Glue console
(couldn't do it with ALTER TABLE …)
I guess this is a bug with Glue crawlers, which do not set these properties
when creating the table.
--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscr...@spark.apache.org
