Yeah, the dot notation works. It works even for arrays. But I am not sure
if it can handle complex hierarchies.
On Mon Dec 08 2014 at 11:55:19 AM Cheng Lian lian.cs@gmail.com wrote:
You may access it via something like SELECT filterIp.element FROM tb,
just like Hive. Or if you’re using
I went through complex hierarchal JSON structures and Spark seems to fail in
querying them no matter what syntax is used.
Hope this helps,
Regards,
Alessandro
On Dec 8, 2014, at 6:05 AM, Raghavendra Pandey raghavendra.pan...@gmail.com
wrote:
Yeah, the dot notation works. It works even
Hi,
I'm generating a Spark SQL table from an offline Json file.
The difficulty is, in the original json file, there is a hierarchical
structure. And, as a result, this is what I get:
scala tb.printSchemaroot |-- budget: double (nullable = true) |-- filterIp:
array (nullable = true) |
Hi,
I'm generating a Spark SQL table from an offline Json file.
The difficulty is, in the original json file, there is a hierarchical
structure. And, as a result, this is what I get:
scala tb.printSchemaroot |-- budget: double (nullable = true) |-- filterIp:
array (nullable = true) |
You may access it via something like |SELECT filterIp.element FROM tb|,
just like Hive. Or if you’re using Spark SQL DSL, you can use
|tb.select(filterIp.element.attr)|.
On 12/8/14 1:08 PM, Xuelin Cao wrote:
Hi,
I'm generating a Spark SQL table from an offline Json file.
The
