[jira] [Commented] (SPARK-24445) Schema in json format for from_json in SQL
[
https://issues.apache.org/jira/browse/SPARK-24445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16497084#comment-16497084
]
Apache Spark commented on SPARK-24445:
--
User 'MaxGekk' has created a pull request for this issue:
https://github.com/apache/spark/pull/21472
> Schema in json format for from_json in SQL
> --
>
> Key: SPARK-24445
> URL: https://issues.apache.org/jira/browse/SPARK-24445
> Project: Spark
> Issue Type: Improvement
> Components: SQL
>Affects Versions: 2.3.0
>Reporter: Maxim Gekk
>Priority: Minor
>
> In Spark 2.3, schema for the from_json function can be specified in JSON
> format in Scala and Python but in SQL. In SQL it is impossible to specify map
> type for example because SQL DDL parser can handle struct type only. Need to
> support schemas in JSON format as it has been already implemented
> [there|https://github.com/apache/spark/blob/branch-2.3/sql/core/src/main/scala/org/apache/spark/sql/functions.scala#L3225-L3229]:
> {code:scala}
> val dataType = try {
> DataType.fromJson(schema)
> } catch {
> case NonFatal(_) => StructType.fromDDL(schema)
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-24445) Schema in json format for from_json in SQL
[
https://issues.apache.org/jira/browse/SPARK-24445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16497075#comment-16497075
]
Maxim Gekk commented on SPARK-24445:
I am working on the ticket at the moment.
> Schema in json format for from_json in SQL
> --
>
> Key: SPARK-24445
> URL: https://issues.apache.org/jira/browse/SPARK-24445
> Project: Spark
> Issue Type: Improvement
> Components: SQL
>Affects Versions: 2.3.0
>Reporter: Maxim Gekk
>Priority: Minor
>
> In Spark 2.3, schema for the from_json function can be specified in JSON
> format in Scala and Python but in SQL. In SQL it is impossible to specify map
> type for example because SQL DDL parser can handle struct type only. Need to
> support schemas in JSON format as it has been already implemented
> [there|https://github.com/apache/spark/blob/branch-2.3/sql/core/src/main/scala/org/apache/spark/sql/functions.scala#L3225-L3229]:
> {code:scala}
> val dataType = try {
> DataType.fromJson(schema)
> } catch {
> case NonFatal(_) => StructType.fromDDL(schema)
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
