> On Nov. 7, 2014, 11:07 p.m., Xuefu Zhang wrote: > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/SparkMapJoinResolver.java, > > line 100 > > <https://reviews.apache.org/r/27627/diff/2/?file=754549#file754549line100> > > > > It seems possible that current is MJwork, right? Are you going to add > > it to the target?
Yes, it's possible. But that MJwork will be a one of which all HTS are already handled, so we can go through it to some HTS for other MJworks. > On Nov. 7, 2014, 11:07 p.m., Xuefu Zhang wrote: > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/SparkMapJoinResolver.java, > > line 115 > > <https://reviews.apache.org/r/27627/diff/2/?file=754549#file754549line115> > > > > Frankly, I'm not 100% following the logic. The diagram has operators > > mixed with works, which makes it hard. But I'm seeing where you're coming > > from. Maybe you can explain to me better in person. Here the operator name (MJ, HTS) means a work contains the operator, so MJ is a BaseWork containing MJ operator, and same for HTS. Yes, I think explaining in person would be better. > On Nov. 7, 2014, 11:07 p.m., Xuefu Zhang wrote: > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/SparkMapJoinResolver.java, > > line 155 > > <https://reviews.apache.org/r/27627/diff/2/?file=754549#file754549line155> > > > > I think there is a separate JIRA handling combining mapjoins, owned by > > Szehon. In my understanding, Szehon's JIRA is try to put MJ operators in the same BaseWork. But, there're some cases that we cannot apply this optimization, and MJ operators will be in different BaseWorks. My work here is to try to put them in the same SparkWork, if there's no dependency among them. - Chao ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27627/#review60403 ----------------------------------------------------------- On Nov. 7, 2014, 6:07 p.m., Chao Sun wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/27627/ > ----------------------------------------------------------- > > (Updated Nov. 7, 2014, 6:07 p.m.) > > > Review request for hive. > > > Bugs: HIVE-8622 > https://issues.apache.org/jira/browse/HIVE-8622 > > > Repository: hive-git > > > Description > ------- > > This is a sub-task of map-join for spark > https://issues.apache.org/jira/browse/HIVE-7613 > This can use the baseline patch for map-join > https://issues.apache.org/jira/browse/HIVE-8616 > > > Diffs > ----- > > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/SparkMapJoinResolver.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/plan/SparkWork.java 66fd6b6 > > Diff: https://reviews.apache.org/r/27627/diff/ > > > Testing > ------- > > > Thanks, > > Chao Sun > >
