[
https://issues.apache.org/jira/browse/SPARK-24643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16527013#comment-16527013
]
Hyukjin Kwon commented on SPARK-24643:
--------------------------------------
[~maxgekk] shall we leave this closed for now?
> from_json should accept an aggregate function as schema
> -------------------------------------------------------
>
> Key: SPARK-24643
> URL: https://issues.apache.org/jira/browse/SPARK-24643
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 2.3.1
> Reporter: Maxim Gekk
> Priority: Minor
>
> Currently, the *from_json()* function accepts only string literals as schema:
> - Checking of schema argument inside of JsonToStructs:
> [https://github.com/apache/spark/blob/b8f27ae3b34134a01998b77db4b7935e7f82a4fe/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala#L530]
> - Accepting only string literal:
> [https://github.com/apache/spark/blob/b8f27ae3b34134a01998b77db4b7935e7f82a4fe/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala#L749-L752]
> JsonToStructs should be modified to accept results of aggregate functions
> like *infer_schema* (see SPARK-24642). It should be possible to write SQL
> like:
> {code:sql}
> select from_json(json_col, infer_schema(json_col)) from json_table
> {code}
> Here is a test case with existing aggregate function - *first()*:
> {code:sql}
> create temporary view schemas(schema) as select * from values
> ('struct<a:int>'),
> ('map<string,int>');
> select from_json('{"a":1}', first(schema)) from schemas;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
