----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27640/#review60052 -----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/SparkMapJoinResolver.java <https://reviews.apache.org/r/27640/#comment101408> Hi Suhas, I was taking a look through the code, I dont think its easy now to identify which is the big-table parent vs small-table parent. There is a HashTableDummyOperator representing the small-table but it only has some basic information. Maybe you know more about it, but was wondering do we need to save the info to a context when we cut the small-table RS from MapJoin in ReduceSinkMapJoinProc? Thanks. - Szehon Ho On Nov. 5, 2014, 8:29 p.m., Suhas Satish wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/27640/ > ----------------------------------------------------------- > > (Updated Nov. 5, 2014, 8:29 p.m.) > > > Review request for hive, Chao Sun, Jimmy Xiang, Szehon Ho, and Xuefu Zhang. > > > Repository: hive-git > > > Description > ------- > > This replaces ReduceSinks with HashTableSinks in smaller tables for a > map-join. But the condition check field to detect map-join is actually being > set in CommonJoinResolver, which doesnt exist yet. We need to decide where is > the right place to populate this field. > > > Diffs > ----- > > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/SparkMapJoinResolver.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/parse/spark/SparkCompiler.java > 795a5d7 > > Diff: https://reviews.apache.org/r/27640/diff/ > > > Testing > ------- > > > Thanks, > > Suhas Satish > >
