AFAIK, ORC uses “hive.optimize.index.filter” hive config to enable predicate pushdown. Can you please try by setting hive.optimize.index.filter to true?
Thanks Prasanth Jayachandran On Nov 7, 2013, at 4:04 PM, Avrilia Floratou <avrilia.flora...@gmail.com> wrote: > Hi all, > > I'm using hive-12. I have a file that contains 10 integer columns stored in > ORC format. The ORC file is zlib compressed and indexing is enabled. > I'm running a simple select count(*) with a predicate of the form (Col1 =0 OR > col2 = 0 etc). The predicate touches all 10 columns but its selectivity is 0 > (none of the records in the file > satisfy the predicate). I have set the hive.optimize.ppd property to true. > For this query basically none of the stripes should be read. However after > the MR job finishes, I can see that the whole file has been read. Why is > that? Is there any other parameter that I should enable for predicate > pushdown/indexing to work? > > Thanks, > Avrilia -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
