-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34059/
-----------------------------------------------------------
(Updated July 10, 2015, 10:17 p.m.)
Review request for hive, Matt McCline and Vikram Dixit Kumaraswamy.
Changes
-------
- Rebase with trunk, looks like some methods in GenTezUtils were converted to
static
- When selecting distributed hash join, the join operator should get
OpTraits/stats set
- For the issue regarding the flattened expressions in the vectorized
rowObjectInspector, change the workaround to un-flatten the object inspector
during JoinUtil.getObjectInspectorsFromEvaluators(). This is still a bit of a
workaround, but only requires a change in 1 place, rather than the 2 changes
needed in the previous solution (having to modify the column names during
vectorized MapJoinOperator, as well as when generating the vectorized
rowObjectInspector in VectorizedBatchUtil)
- In the reducer, only the big table's input source should be vectorized
Bugs: HIVE-10673
https://issues.apache.org/jira/browse/HIVE-10673
Repository: hive-git
Description
-------
Reduce-side hash join (using MapJoinOperator), where the Tez inputs to the
reducer are unsorted.
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 39477d6
itests/src/test/resources/testconfiguration.properties 97715fc
ql/src/java/org/apache/hadoop/hive/ql/exec/JoinUtil.java 7b57550
ql/src/java/org/apache/hadoop/hive/ql/exec/MapJoinOperator.java 15cafdd
ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java d7f1b42
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/KeyValuesAdapter.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/KeyValuesFromKeyValue.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/KeyValuesFromKeyValues.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/ReduceRecordProcessor.java
545d7c6
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/ReduceRecordSource.java
7d79e87
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorMapJoinOperator.java
e9bd44a
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/VectorMapJoinCommonOperator.java
4c8c4b1
ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConvertJoinMapJoin.java
5a87bd6
ql/src/java/org/apache/hadoop/hive/ql/optimizer/MapJoinProcessor.java 4d84f0f
ql/src/java/org/apache/hadoop/hive/ql/optimizer/ReduceSinkMapJoinProc.java
bca91dd
ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezProcContext.java f474eae
ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezUtils.java 93ad145
ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezWork.java 6b3e19d
ql/src/java/org/apache/hadoop/hive/ql/plan/BaseWork.java fa697ef
ql/src/java/org/apache/hadoop/hive/ql/plan/CommonMergeJoinDesc.java f9c34cb
ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeDescUtils.java fb3c4a3
ql/src/java/org/apache/hadoop/hive/ql/plan/MapJoinDesc.java cee9100
ql/src/java/org/apache/hadoop/hive/ql/plan/ReduceWork.java a78a92e
ql/src/test/queries/clientpositive/tez_dynpart_hashjoin_1.q PRE-CREATION
ql/src/test/queries/clientpositive/tez_dynpart_hashjoin_2.q PRE-CREATION
ql/src/test/queries/clientpositive/tez_vector_dynpart_hashjoin_1.q
PRE-CREATION
ql/src/test/queries/clientpositive/tez_vector_dynpart_hashjoin_2.q
PRE-CREATION
ql/src/test/results/clientpositive/tez/tez_dynpart_hashjoin_1.q.out
PRE-CREATION
ql/src/test/results/clientpositive/tez/tez_dynpart_hashjoin_2.q.out
PRE-CREATION
ql/src/test/results/clientpositive/tez/tez_vector_dynpart_hashjoin_1.q.out
PRE-CREATION
ql/src/test/results/clientpositive/tez/tez_vector_dynpart_hashjoin_2.q.out
PRE-CREATION
Diff: https://reviews.apache.org/r/34059/diff/
Testing
-------
q-file tests added
Thanks,
Jason Dere
