-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/857/
-----------------------------------------------------------
(Updated 2011-06-10 06:35:32.125295)
Review request for hive and John Sichi.
Changes
-------
Based on a discussion with yongqian, I re-implemented the predicate
decomposition into two steps, computing the overall residual predicate from the
union of all columns in the available indexes, and then computing the
predicates to apply to each index individually. Additionally I have also
extended the functionality to pass in partition columns to allowColumnNames and
added/extended the testcases to check that partition predicates are propagated
correctly. This required adding a check in IndexWhereProcessor.java that the
correct FilterOperator was passed to the process(...) method (apparently a
duplicate FilterOperator that does not have the entire predicate gets created).
Summary
-------
Add support for generating index queries to support automatic usage of bitmap
indexes. This required changing the interface to the IndexHandlers to support
accepting queries on multiple indexes. The compact indexes were modified to use
this new interface as well, although no functional changes were made to how
they work. Only supports AND predicates right now, but it should be possibly to
extend the BitmapQuery interface defined in this patch to easily support OR
predicates as well. Currently benchmarking these changes on a test cluster.
This addresses bug HIVE-2036.
https://issues.apache.org/jira/browse/HIVE-2036
Diffs (updated)
-----
ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java 4fba845
ql/src/java/org/apache/hadoop/hive/ql/index/HiveIndexHandler.java e5ee183
ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapIndexHandler.java
af9d7b1
ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapInnerQuery.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapOuterQuery.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapQuery.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/index/compact/CompactIndexHandler.java
56e7609
ql/src/java/org/apache/hadoop/hive/ql/io/HiveFileFormatUtils.java d64e88b
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereProcessor.java
268560d
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereTaskDispatcher.java
0873e1a
ql/src/test/queries/clientpositive/index_auto_partitioned.q 5f92f04
ql/src/test/queries/clientpositive/index_bitmap_auto.q PRE-CREATION
ql/src/test/queries/clientpositive/index_bitmap_auto_partitioned.q
PRE-CREATION
ql/src/test/results/clientpositive/index_auto_partitioned.q.out 05cc84a
ql/src/test/results/clientpositive/index_bitmap_auto.q.out PRE-CREATION
ql/src/test/results/clientpositive/index_bitmap_auto_partitioned.q.out
PRE-CREATION
Diff: https://reviews.apache.org/r/857/diff
Testing
-------
Passes unit tests, additional testcase to test automatic bitmap indexing
index_bitmap_auto.q was also added to the TestCliDriver suite. Currently
benchmarking changes on a test cluster.
Thanks,
Syed
