Huon Wilson created SPARK-26964:
-----------------------------------
Summary: to_json/from_json do not match JSON spec due to not
supporting scalars
Key: SPARK-26964
URL: https://issues.apache.org/jira/browse/SPARK-26964
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 2.4.0, 2.3.2
Reporter: Huon Wilson
Spark SQL's {{to_json}} and {{from_json}} currently support arrays and objects,
but not the scalar/primitive types. This doesn't match the JSON spec on
https://www.json.org: a JSON document ({{json: element}}) consists of a value
surrounded by whitespace ({{element: ws value ws}}), where a value is an object
or array _or_ a number or string etc.:
{code:none}
value
object
array
string
number
"true"
"false"
"null"
{code}
Having {{to_json}} and {{from_json}} support scalars would make them flexible
enough for a library I'm working on, where an arbitrary (user-supplied) column
needs to be turned into JSON.
This is related to SPARK-24391 and SPARK-25252, which added support for arrays
of scalars.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
