Github user patrickmcgloin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21671#discussion_r199330240
--- Diff: python/pyspark/sql/functions.py ---
@@ -2163,9 +2163,9 @@ def json_tuple(col, *fields):
@since(2.1)
def from_json(col, schema, options={}):
"""
- Parses a column containing a JSON string into a :class:`MapType` with
:class:`StringType`
- as keys type, :class:`StructType` or :class:`ArrayType` of
:class:`StructType`\\s with
- the specified schema. Returns `null`, in the case of an unparseable
string.
+ Parses a column containing a JSON string into a :class:`MapType`,
:class:`StructType`
+ or :class:`ArrayType` of :class:`StructType`\\s with the specified
schema. Returns
+ `null`, in the case of an unparseable string.
--- End diff --
I think that other basic types (e.g Int, Long, etc) were already supported
along with String. As long as they can be directly converted from String to
that type specified in the schema then they were ok. What we have added here
is parsing on the DateType and TimestampType. I don't think complex types
would ever be supported. And I don't think other basic types need the parsing
that date/times do.
Do you think we should specify which types are supported as keys?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
