-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30388/
-----------------------------------------------------------
Review request for hive and Xuefu Zhang.
Bugs: HIVE-9103
https://issues.apache.org/jira/browse/HIVE-9103
Repository: hive-git
Description
-------
This patch adds backup task to map join task. The backup task, which uses
common join, will be triggered
in case the mapjoin task failed.
Note that, no matter how many map joins there are in the SparkTask, we will
only generate one backup task.
This means that if the original task failed at the very last map join, the
whole task will be re-executed.
The handling of backup task is a little bit different from what MR does, mostly
because we convert JOIN to
MAPJOIN during the operator plan optimization phase, at which time no task/work
exist yet. In the patch, we
cloned the whole operator tree before the JOIN operator is converted. The
operator tree will be processed
and generate a separate work tree for a separate backup SparkTask.
Diffs
-----
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/SparkMapJoinResolver.java
69004dc
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/StageIDsRearranger.java
79c3e02
ql/src/java/org/apache/hadoop/hive/ql/optimizer/spark/SparkJoinOptimizer.java
d57ceff
ql/src/java/org/apache/hadoop/hive/ql/optimizer/spark/SparkMapJoinOptimizer.java
9ff47c7
ql/src/java/org/apache/hadoop/hive/ql/optimizer/spark/SparkSortMergeJoinFactory.java
6e0ac38
ql/src/java/org/apache/hadoop/hive/ql/parse/ParseContext.java b838bff
ql/src/java/org/apache/hadoop/hive/ql/parse/spark/GenSparkProcContext.java
773cfbd
ql/src/java/org/apache/hadoop/hive/ql/parse/spark/OptimizeSparkProcContext.java
f7586a4
ql/src/java/org/apache/hadoop/hive/ql/parse/spark/SparkCompiler.java 3a7477a
ql/src/java/org/apache/hadoop/hive/ql/plan/TableScanDesc.java 0e85990
ql/src/test/results/clientpositive/spark/auto_join25.q.out ab01b8a
Diff: https://reviews.apache.org/r/30388/diff/
Testing
-------
auto_join25.q
Thanks,
Chao Sun
