[
https://issues.apache.org/jira/browse/HIVE-2036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13045718#comment-13045718
]
[email protected] commented on HIVE-2036:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/857/
-----------------------------------------------------------
(Updated 2011-06-08 00:22:37.292935)
Review request for hive and John Sichi.
Changes
-------
Addressed comments. Still does not propagate partition predicates to every
single index sub-query, but it does ensure that predicates are only applied to
indexes for which there are matching columns. After looking at the behavior of
CompactIndexHandler on partitioned tables (and in testcase
index_auto_partitioned.q) I can't quite see how the CompactIndexHandler
identifies and propagates partitioning predicates correctly.
Summary
-------
Add support for generating index queries to support automatic usage of bitmap
indexes. This required changing the interface to the IndexHandlers to support
accepting queries on multiple indexes. The compact indexes were modified to use
this new interface as well, although no functional changes were made to how
they work. Only supports AND predicates right now, but it should be possibly to
extend the BitmapQuery interface defined in this patch to easily support OR
predicates as well. Currently benchmarking these changes on a test cluster.
This addresses bug HIVE-2036.
https://issues.apache.org/jira/browse/HIVE-2036
Diffs (updated)
-----
ql/src/test/queries/clientpositive/index_bitmap_auto_partitioned.q
PRE-CREATION
ql/src/test/results/clientpositive/index_bitmap_auto.q.out PRE-CREATION
ql/src/test/queries/clientpositive/index_bitmap_auto.q PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereTaskDispatcher.java
0873e1a
ql/src/java/org/apache/hadoop/hive/ql/index/compact/CompactIndexHandler.java
56e7609
ql/src/java/org/apache/hadoop/hive/ql/io/HiveFileFormatUtils.java d64e88b
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/index/IndexWhereProcessor.java
268560d
ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapQuery.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapInnerQuery.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapOuterQuery.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java 4fba845
ql/src/java/org/apache/hadoop/hive/ql/index/HiveIndexHandler.java e5ee183
ql/src/java/org/apache/hadoop/hive/ql/index/bitmap/BitmapIndexHandler.java
af9d7b1
ql/src/test/results/clientpositive/index_bitmap_auto_partitioned.q.out
PRE-CREATION
Diff: https://reviews.apache.org/r/857/diff
Testing
-------
Passes unit tests, additional testcase to test automatic bitmap indexing
index_bitmap_auto.q was also added to the TestCliDriver suite. Currently
benchmarking changes on a test cluster.
Thanks,
Syed
> Update bitmap indexes for automatic usage
> -----------------------------------------
>
> Key: HIVE-2036
> URL: https://issues.apache.org/jira/browse/HIVE-2036
> Project: Hive
> Issue Type: Improvement
> Components: Indexing
> Affects Versions: 0.8.0
> Reporter: Russell Melick
> Assignee: Syed S. Albiz
> Attachments: HIVE-2036.1.patch
>
>
> HIVE-1644 will provide automatic usage of indexes, and HIVE-1803 adds bitmap
> index support. The bitmap code will need to be extended after it is
> committed to enable automatic use of indexing. Most work will be focused in
> the BitmapIndexHandler, which needs to generate the re-entrant QL index
> query. There may also be significant work in the IndexPredicateAnalyzer to
> support predicates with OR's, instead of just AND's as it is currently.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
