Re: Nested types in ORC

2014-09-09 Thread Abhishek Agarwal
Thanks Prasanth. Does it also mean that a query reading nested.k column will invariably read nested.v as well even if nested.v column in not used in the query? On Mon, Sep 8, 2014 at 11:29 PM, Prasanth Jayachandran pjayachand...@hortonworks.com wrote: Hi ORC stores nested fields as separate

Re: Nested types in ORC

2014-09-09 Thread Prasanth Jayachandran
Yes. It does now. Thanks Prasanth Jayachandran On Sep 9, 2014, at 12:30 AM, Abhishek Agarwal abhishc...@gmail.com wrote: Thanks Prasanth. Does it also mean that a query reading nested.k column will invariably read nested.v as well even if nested.v column in not used in the query? On

Nested types in ORC

2014-09-08 Thread Abhishek Agarwal
Hi all, I have few questions with regards to nested columns in Hive. How does ORC internally stores the complex types such as a struct? Are the nested fields stored as separate columns or is the whole struct is serialized as one column? Is predicate pushdown supported for queries which access

Re: Nested types in ORC

2014-09-08 Thread Prasanth Jayachandran
Hi ORC stores nested fields as separate columns. For example: The following table create table orc_nested (key string, nested structk:string,v:string, zip long) stored as orc; will be flattened and stored as separated columns like below key, nested, nested.k, nested.v, zip you can have a look