----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33251/#review80969 -----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/exec/spark/HashTableLoader.java <https://reviews.apache.org/r/33251/#comment131153> 1. For clarity, it might be good to put this in a separate private method. 2. Does it work if we just synchronize on mapJoinTables[pos]? ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkRecordHandler.java <https://reviews.apache.org/r/33251/#comment131156> Method naming, see below. ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkUtilities.java <https://reviews.apache.org/r/33251/#comment131154> Using thread-local makes me a little nervous, but let's discuss about this offline. ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkUtilities.java <https://reviews.apache.org/r/33251/#comment131155> The method name suggests no indication of a side effect of setting thread local value. We'd better put this outside of this method. In addition, the method name seems also a little confusing in that it suggests cleanup is for sure but in fact it's conditional. - Xuefu Zhang On April 21, 2015, 1:37 a.m., Jimmy Xiang wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/33251/ > ----------------------------------------------------------- > > (Updated April 21, 2015, 1:37 a.m.) > > > Review request for hive, Chao Sun, Szehon Ho, and Xuefu Zhang. > > > Bugs: HIVE-10302 > https://issues.apache.org/jira/browse/HIVE-10302 > > > Repository: hive-git > > > Description > ------- > > Cached the small table containter so that mapjoin tasks can use it if the > task is executed on the same Spark executor. > The cache is released right before the next job after the mapjoin job is done. > > > Diffs > ----- > > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/HashTableLoader.java > fe108c4 > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkPlanGenerator.java > 3f240f5 > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkRecordHandler.java > 97b3471 > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkUtilities.java > 72ab913 > > Diff: https://reviews.apache.org/r/33251/diff/ > > > Testing > ------- > > Ran several queries in live cluster. ptest pending. > > > Thanks, > > Jimmy Xiang > >
