[
https://issues.apache.org/jira/browse/SPARK-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14105221#comment-14105221
]
Hanwei Jin commented on SPARK-2096:
---
I think i almost have solved the issue.
I have passed the test case in JsonSuite (Complex field and type inferring
(Ignored)) which is ignored, by a little modified.
modified test part :
checkAnswer(
sql(select arrayOfStruct.field1, arrayOfStruct.field2 from jsonTable),
(Seq(true, false, null), Seq(str1, null, null)) :: Nil
)
However, another question is repeated nested structure is a problem, like
arrayOfStruct.field1.arrayOfStruct.field1 or
arrayOfStruct[0].field1.arrayOfStruct[0].field1
I plan to ignore this problem and try to add select arrayOfStruct.field1,
arrayOfStruct.field2 from jsonTable where arrayOfStruct.field1==true
Besides, my friend anyweil (Wei Li) solved the problem of arrayOfStruct.field1
and its Filter part( means where parsing).
I am fresh here but will continue working on spark :)
Correctly parse dot notations for accessing an array of structs
---
Key: SPARK-2096
URL: https://issues.apache.org/jira/browse/SPARK-2096
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 1.0.0
Reporter: Yin Huai
Priority: Minor
Labels: starter
For example, arrayOfStruct is an array of structs and every element of this
array has a field called field1. arrayOfStruct[0].field1 means to access
the value of field1 for the first element of arrayOfStruct, but the SQL
parser (in sql-core) treats field1 as an alias. Also,
arrayOfStruct.field1 means to access all values of field1 in this array
of structs and the returns those values as an array. But, the SQL parser
cannot resolve it.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org