Maxim Gekk created SPARK-24027:
----------------------------------
Summary: Support MapType(StringType, DataType) as root type by
from_json
Key: SPARK-24027
URL: https://issues.apache.org/jira/browse/SPARK-24027
Project: Spark
Issue Type: New Feature
Components: SQL
Affects Versions: 2.3.0
Reporter: Maxim Gekk
Currently, *MapType* is not supported by the *from_json* function as the root
type. For example, the following code doesn't work on Spark 2.3:
{code}
scala> import org.apache.spark.sql.types._
import org.apache.spark.sql.types._
scala> val schema = MapType(StringType, IntegerType)
schema: org.apache.spark.sql.types.MapType =
MapType(StringType,IntegerType,true)
scala> val in = Seq("""{"a": 1, "b": 2, "c": 3}""").toDS()
in: org.apache.spark.sql.Dataset[String] = [value: string]
scala> in.select(from_json($"value", schema, Map[String, String]())).collect()
org.apache.spark.sql.AnalysisException: cannot resolve 'jsontostructs(`value`)'
due to data type mismatch: Input schema map<string,int> must be a struct or an
array of structs.
{code}
Purpose of the ticket is to support _MapType with StringType as keys type_.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
