Hi ,
I am trying to read Nested Avro data in Spark 1.3 using DataFrames.
I need help to retrieve the Inner element data in the Structure below.
Below is the schema when I enter df.printSchema :
|-- THROTTLING_PERCENTAGE: double (nullable = false)
|-- IMPRESSION_TYPE: string (nullable = false)
Typically you would use . notation to access, same way you would access a
map.
On 12 May 2015 00:06, Ashish Kumar Singh ashish23...@gmail.com wrote:
Hi ,
I am trying to read Nested Avro data in Spark 1.3 using DataFrames.
I need help to retrieve the Inner element data in the Structure below.
Since there is an array here you are probably looking for HiveQL's LATERAL
VIEW explode
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+LateralView
.
On Mon, May 11, 2015 at 7:12 AM, ayan guha guha.a...@gmail.com wrote:
Typically you would use . notation to access, same way you
Had the same question on stackoverflow recently
http://stackoverflow.com/questions/30008127/how-to-read-a-nested-collection-in-spark
Lomig Mégard had a detailed answer of how to do this without using LATERAL
VIEW.
On Mon, May 11, 2015 at 8:05 AM, Ashish Kumar Singh ashish23...@gmail.com
wrote:
