Yeah, the dot notation works. It works even for arrays. But I am not sure
if it can handle complex hierarchies.
On Mon Dec 08 2014 at 11:55:19 AM Cheng Lian <lian.cs....@gmail.com> wrote:
> You may access it via something like SELECT filterIp.element FROM tb,
> just like Hive. Or if you’re using Spark SQL DSL, you can use
> tb.select("filterIp.element".attr).
>
> On 12/8/14 1:08 PM, Xuelin Cao wrote:
>
>
> Hi,
>
> I'm generating a Spark SQL table from an offline Json file.
>
> The difficulty is, in the original json file, there is a
> hierarchical structure. And, as a result, this is what I get:
>
> scala> tb.printSchema
> root
> |-- budget: double (nullable = true)
> * |-- filterIp: array (nullable = true)*
> * | |-- element: string (containsNull = false)*
> |-- status: integer (nullable = true)
> |-- third_party: integer (nullable = true)
> |-- userId: integer (nullable = true)
>
> As you may have noticed, the table schema is with a hierarchical
> structure ("element" field is a sub-field under the "filterIp" field).
> Then, my question is, how do I access the "element" field with SQL?
>
>
>
>
