Hi,
Apparently is it is possible to query nested json using spark SQL, but ,
mainly due to lack of proper documentation/examples, I did not manage to
make it working. I do appreciate if you could point me to any example or
help with this issue,
Here is my code:
val anotherPeopleRDD = sc.parallelize(
"""{
"attributes": [
{
"data": {
"gender": "woman"
},
"section": "Economy",
"collectApp": "web",
"id": 1409064792512
}
]
}""" :: Nil)
val anotherPeople = sqlContext.jsonRDD(anotherPeopleRDD)
anotherPeople.registerTempTable("people")
val query_people = sqlContext.sql("select attributes[0].collectApp from
people")
query_people.foreach(println)
But instead of getting "Web" as print out, I am getting the following:
[[web,[woman],1409064792512, Economy]]
thanks,
/shahab