Re: Querying nested struct fields

I tried the same, didn't work :( scala> hc.sql("select _1.item_id from agg_imps_df limit 10").collect() 15/11/10 14:30:41 INFO parse.ParseDriver: Parsing command: select _1.item_id from agg_imps_df limit 10 org.apache.spark.sql.AnalysisException: missing \' at 'from' near ''; line 1 pos 23

Re: Querying nested struct fields

Use a `.`: hc.sql("select _1.item_id from agg_imps_df limit 10").collect() On Tue, Nov 10, 2015 at 11:24 AM, pratik khadloya wrote: > Hello, > > I just saved a PairRDD as a table, but i am not able to query it > correctly. The below and other variations does not seem to

Querying nested struct fields

Hello, I just saved a PairRDD as a table, but i am not able to query it correctly. The below and other variations does not seem to work. scala> hc.sql("select * from agg_imps_df").printSchema() |-- _1: struct (nullable = true) ||-- item_id: long (nullable = true) ||-- flight_id: long

Re: Querying nested struct fields

Oh sorry _1 is not a valid hive identifier, you need to use backticks to escape it: Seq(((1, 2), 2)).toDF().registerTempTable("test") sql("SELECT `_1`.`_1` FROM test") On Tue, Nov 10, 2015 at 11:31 AM, pratik khadloya wrote: > I tried the same, didn't work :( > > scala>

Re: Querying nested struct fields

That worked!! Thanks a lot Michael. ~Pratik On Tue, Nov 10, 2015 at 12:02 PM Michael Armbrust wrote: > Oh sorry _1 is not a valid hive identifier, you need to use backticks to > escape it: > > Seq(((1, 2), 2)).toDF().registerTempTable("test") > sql("SELECT `_1`.`_1`