planga82 commented on pull request #32599: URL: https://github.com/apache/spark/pull/32599#issuecomment-847322940
> Just to clarify, it works iff users specify the explicit map type. Schema inference won't infer such schema. Yes, Schema inference won't infer such schema We have clear then that it is not a Bug, as @cloud-fan and @viirya say, the keys of a Json are always of type String, so this would be an improvement. I think that since we allow write operations of type `df.withColumn(colName, from_json(col(colName), MapType(DoubleType, StringType))).write.json(jsonDir)` it is more consistent to support the reverse operation `spark.read.schema(StructType(Seq(StructField(DoubleType, dataType)))).json(jsonDir)` Anyway, if we decide to not support this, I think we need to improve the error message -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
