-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7126/
-----------------------------------------------------------
(Updated Nov. 19, 2012, 7:51 p.m.)
Review request for hive.
Changes
-------
Correlation optimizer will guess which join operators at the bottom (input
tables are not intermediate tables) will be optimized by auto join convert and
ignore those join operators in the optimization of correlation optimizer.
Description
-------
This optimizer exploits intra-query correlations and merges multiple correlated
MapReduce jobs into one jobs. Open a new request since I have been working on
hive-git.
This addresses bug HIVE-2206.
https://issues.apache.org/jira/browse/HIVE-2206
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 9fa9525
conf/hive-default.xml.template f332f3a
ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/OperatorType.java
7c4c413
ql/src/java/org/apache/hadoop/hive/ql/exec/BaseReduceSinkOperator.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/CorrelationCompositeOperator.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/CorrelationLocalSimulativeReduceSinkOperator.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/CorrelationReducerDispatchOperator.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/ExecReducer.java 18a9bd2
ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java 46daeb2
ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java 68302f8
ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorFactory.java 0c22141
ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java 919a140
ql/src/java/org/apache/hadoop/hive/ql/exec/TableScanOperator.java 1469325
ql/src/java/org/apache/hadoop/hive/ql/optimizer/CorrelationOptimizer.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/optimizer/CorrelationOptimizerUtils.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java edde378
ql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java d1555e2
ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java 2bf284d
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 330aa52
ql/src/java/org/apache/hadoop/hive/ql/plan/BaseReduceSinkDesc.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/plan/CorrelationCompositeDesc.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/plan/CorrelationLocalSimulativeReduceSinkDesc.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/plan/CorrelationReducerDispatchDesc.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/plan/MapredWork.java 5a9f064
ql/src/java/org/apache/hadoop/hive/ql/plan/ReduceSinkDesc.java b33d616
ql/src/java/org/apache/hadoop/hive/ql/plan/TableScanDesc.java 9a95efd
ql/src/test/org/apache/hadoop/hive/ql/exec/TestExecDriver.java 6f8bc47
ql/src/test/queries/clientpositive/correlationoptimizer1.q PRE-CREATION
ql/src/test/queries/clientpositive/correlationoptimizer2.q PRE-CREATION
ql/src/test/queries/clientpositive/correlationoptimizer3.q PRE-CREATION
ql/src/test/queries/clientpositive/correlationoptimizer4.q PRE-CREATION
ql/src/test/queries/clientpositive/correlationoptimizer5.q PRE-CREATION
ql/src/test/results/clientpositive/correlationoptimizer1.q.out PRE-CREATION
ql/src/test/results/clientpositive/correlationoptimizer2.q.out PRE-CREATION
ql/src/test/results/clientpositive/correlationoptimizer3.q.out PRE-CREATION
ql/src/test/results/clientpositive/correlationoptimizer4.q.out PRE-CREATION
ql/src/test/results/clientpositive/correlationoptimizer5.q.out PRE-CREATION
ql/src/test/results/compiler/plan/groupby1.q.xml cd0d6e4
ql/src/test/results/compiler/plan/groupby2.q.xml 7b07f02
ql/src/test/results/compiler/plan/groupby3.q.xml a6a1986
ql/src/test/results/compiler/plan/groupby5.q.xml 25e3583
Diff: https://reviews.apache.org/r/7126/diff/
Testing
-------
All tests pass.
Thanks,
Yin Huai
