Thanks Prasanth. Does it also mean that a query reading nested.k column will invariably read nested.v as well even if nested.v column in not used in the query?
On Mon, Sep 8, 2014 at 11:29 PM, Prasanth Jayachandran < pjayachand...@hortonworks.com> wrote: > Hi > > ORC stores nested fields as separate columns. For example: The following > table > create table orc_nested (key string, nested struct<k:string,v:string>, zip > long) stored as orc; > will be flattened and stored as separated columns like below > key, nested, nested.k, nested.v, zip > > you can have a look at the structure of ORC files using "hive > —orcfiledump” utility. > > With regard to your next question, predicate pushdown is not supported for > complex types at this point. There is a JIRA already for supporting it > https://issues.apache.org/jira/browse/HIVE-7214 > > At this point, schema 2 will make you enable predicate pushdown. The > performance difference depends mainly on the data layout/if column is > sorted or not. > > Thanks > Prasanth Jayachandran > > On Sep 8, 2014, at 6:16 AM, Abhishek Agarwal <abhishc...@gmail.com> wrote: > > Hi all, > I have few questions with regards to nested columns in Hive. > > How does ORC internally stores the complex types such as a struct? Are > the nested fields stored as separate columns or is the whole struct is > serialized as one column? > > > Is predicate pushdown supported for queries which access nested columns? > In general, is there a significant performance difference in following > schemas with regards to query execution and storage? > > Schema1: > > { > string a; > struct b { > string b1; > string b2; > } > } > > Schema 2: > { > string a; > string b.b1; > string b.b2; > } > > -- > Regards, > Abhishek Agarwal > > > > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity > to which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. -- Regards, Abhishek Agarwal
