----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64688/#review197216 -----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/CustomPartitionVertex.java Line 548 (original), 579 (patched) <https://reviews.apache.org/r/64688/#comment277352> If a bucket file is missing in the list of files, then bucketNum < numBuckets .. so this will trigger the fallback loop below? - Jason Dere On Feb. 10, 2018, 12:41 a.m., Deepak Jaiswal wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64688/ > ----------------------------------------------------------- > > (Updated Feb. 10, 2018, 12:41 a.m.) > > > Review request for hive, Ashutosh Chauhan and Jason Dere. > > > Repository: hive-git > > > Description > ------- > > Bucket based Join : Handle buckets with no splits. > > The current logic in CustomPartitionVertex assumes that there is a split for > each bucket whereas in Tez, we can have no splits for empty buckets. > Also falls back to reduceside join if small table has more buckets than big > table. > > Disallow loading files in bucketed tables if the file name format is not like > 000000_0, 000001_0_copy_1 etc. > > > Diffs > ----- > > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/CustomPartitionVertex.java > 26afe90faa > > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/CustomVertexConfiguration.java > ef5e7edcd6 > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/DagUtils.java 9885038588 > ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConvertJoinMapJoin.java > dc698c8de8 > ql/src/java/org/apache/hadoop/hive/ql/parse/LoadSemanticAnalyzer.java > 54f5bab6de > ql/src/test/queries/clientpositive/auto_sortmerge_join_16.q 8216b538c2 > ql/src/test/results/clientpositive/llap/auto_sortmerge_join_16.q.out > 91408df129 > ql/src/test/results/clientpositive/spark/auto_sortmerge_join_16.q.out_spark > 91408df129 > > > Diff: https://reviews.apache.org/r/64688/diff/4/ > > > Testing > ------- > > > Thanks, > > Deepak Jaiswal > >
