Hi Shahab,
Can you try to use HiveContext? Its should work in 1.1. For SQLContext,
this issues was not fixed in 1.1 and you need to use master branch at the
moment.
Thanks,
Yin
On Sun, Oct 12, 2014 at 5:20 PM, shahab <shahab.mok...@gmail.com> wrote:
> Hi,
>
> Apparently is it is possible to query nested json using spark SQL, but ,
> mainly due to lack of proper documentation/examples, I did not manage to
> make it working. I do appreciate if you could point me to any example or
> help with this issue,
>
> Here is my code:
>
> val anotherPeopleRDD = sc.parallelize(
>
> """{
>
> "attributes": [
>
> {
>
> "data": {
>
> "gender": "woman"
>
> },
>
> "section": "Economy",
>
> "collectApp": "web",
>
> "id": 1409064792512
>
> }
>
> ]
>
> }""" :: Nil)
>
> val anotherPeople = sqlContext.jsonRDD(anotherPeopleRDD)
>
> anotherPeople.registerTempTable("people")
>
> val query_people = sqlContext.sql("select attributes[0].collectApp
> from people")
>
> query_people.foreach(println)
>
> But instead of getting "Web" as print out, I am getting the following:
>
> [[web,[woman],1409064792512, Economy]]
>
>
>
> thanks,
>
> /shahab
>
>
>
