Ewan Leith created SPARK-10947:
----------------------------------
Summary: With schema inference from JSON into a Dataframe, add
option to infer all primitive object types as strings
Key: SPARK-10947
URL: https://issues.apache.org/jira/browse/SPARK-10947
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 1.5.1
Reporter: Ewan Leith
Priority: Minor
Currently, when a schema is inferred from a JSON file using
sqlContext.read.json, the primitive object types are inferred as string, long,
boolean, etc.
However, if the inferred type is too specific (JSON obviously does not enforce
types itself), this causes issues with merging dataframe schemas.
Instead, we would like an option in the JSON inferField function to treat all
primitive objects as strings.
We'll create and submit a pull request for this for review.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
