[ https://issues.apache.org/jira/browse/SPARK-24027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16444724#comment-16444724 ]
Apache Spark commented on SPARK-24027: -------------------------------------- User 'MaxGekk' has created a pull request for this issue: https://github.com/apache/spark/pull/21108 > Support MapType(StringType, DataType) as root type by from_json > --------------------------------------------------------------- > > Key: SPARK-24027 > URL: https://issues.apache.org/jira/browse/SPARK-24027 > Project: Spark > Issue Type: New Feature > Components: SQL > Affects Versions: 2.3.0 > Reporter: Maxim Gekk > Priority: Major > > Currently, *MapType* is not supported by the *from_json* function as the root > type. For example, the following code doesn't work on Spark 2.3: > {code} > scala> import org.apache.spark.sql.types._ > import org.apache.spark.sql.types._ > scala> val schema = MapType(StringType, IntegerType) > schema: org.apache.spark.sql.types.MapType = > MapType(StringType,IntegerType,true) > scala> val in = Seq("""{"a": 1, "b": 2, "c": 3}""").toDS() > in: org.apache.spark.sql.Dataset[String] = [value: string] > scala> in.select(from_json($"value", schema, Map[String, String]())).collect() > org.apache.spark.sql.AnalysisException: cannot resolve > 'jsontostructs(`value`)' due to data type mismatch: Input schema > map<string,int> must be a struct or an array of structs. > {code} > Purpose of the ticket is to support _MapType with StringType as keys type_. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org