> On May 27, 2015, 10:13 p.m., Xuefu Zhang wrote: > > common/src/java/org/apache/hadoop/hive/conf/HiveConf.java, line 2062 > > <https://reviews.apache.org/r/34455/diff/3/?file=972428#file972428line2062> > > > > Sorry for pointing this out late. I'm not certain if it's a good idea > > to expose these two configurations. Also this introduces a change of > > behavior. For now, can we get rid of them and change the persistency level > > back to MEM+DISK? > > > > We can come back to revisit this later on. At this moment, I don't feel > > confident to make the call. > > chengxiang li wrote: > persistent to MEM + DISK may hurt the performance in certain cases, i > think at least we should have a switch to open/close this optimization, > > Xuefu Zhang wrote: > Agreed. However, before we find out more about in what cases this helps > or hurts, I think it's better we keep the existing behavior. This doesn't > prevent us from adding a flag later on. > > chengxiang li wrote: > Ok, i would remove these configurations from patch in temp, we can > discuss later when we got more knowledge about it.
Please feel free to create a followup JIRA to do more research. We can try different data sizes and persistancy levels to see the result. At that time, we can decide if it makes sense to introduce configurations. Thanks. - Xuefu ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34455/#review85451 ----------------------------------------------------------- On May 28, 2015, 3:30 a.m., chengxiang li wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/34455/ > ----------------------------------------------------------- > > (Updated May 28, 2015, 3:30 a.m.) > > > Review request for hive, Chao Sun, Jimmy Xiang, and Xuefu Zhang. > > > Bugs: HIVE-10550 > https://issues.apache.org/jira/browse/HIVE-10550 > > > Repository: hive-git > > > Description > ------- > > see jira description > > > Diffs > ----- > > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/CacheTran.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/MapTran.java 2170243 > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/ReduceTran.java e60dfac > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkPlan.java ee5c78a > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkPlanGenerator.java > 3f240f5 > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkUtilities.java > e6c845c > > Diff: https://reviews.apache.org/r/34455/diff/ > > > Testing > ------- > > > Thanks, > > chengxiang li > >
