batch processing partition pruning process
------------------------------------------
Key: HIVE-2050
URL: https://issues.apache.org/jira/browse/HIVE-2050
Project: Hive
Issue Type: Sub-task
Reporter: Ning Zhang
For partition predicates that cannot be pushed down to JDO filtering
(HIVE-2049), we should fall back to the old approach of listing all partition
names first and use Hive's expression evaluation engine to select the correct
partitions. Then the partition pruner should hand Hive a list of partition
names and return a list of Partition Object (this should be added to the Hive
API).
A possible optimization is that the the partition pruner should give Hive a set
of ranges of partition names (say [ts=01, ts=11], [ts=20, ts=24]), and the JDO
query should be formulated as range queries. Range queries are possible because
the first step list all partition names in sorted order. It's easy to come up
with a range and it is guaranteed that the JDO range query results should be
equivalent to the query with a list of partition names.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
