Re: Review Request 27627: Split map-join plan into 2 SparkTasks in 3 stages [Spark Branch]

2014-11-09 Thread Xuefu Zhang
On Nov. 8, 2014, 3:15 p.m., Xuefu Zhang wrote: ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/SparkMapJoinResolver.java, line 214 https://reviews.apache.org/r/27627/diff/3/?file=754597#file754597line214 This assumes that result SparkWorks will be linearly dependent on

Re: Review Request 27627: Split map-join plan into 2 SparkTasks in 3 stages [Spark Branch]

2014-11-09 Thread Chao Sun
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27627/ --- (Updated Nov. 9, 2014, 10:39 p.m.) Review request for hive. Changes ---

Re: Review Request 27627: Split map-join plan into 2 SparkTasks in 3 stages [Spark Branch]

2014-11-09 Thread Chao Sun
On Nov. 8, 2014, 3:15 p.m., Xuefu Zhang wrote: ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/SparkMapJoinResolver.java, line 214 https://reviews.apache.org/r/27627/diff/3/?file=754597#file754597line214 This assumes that result SparkWorks will be linearly dependent on

Re: Review Request 27627: Split map-join plan into 2 SparkTasks in 3 stages [Spark Branch]

2014-11-09 Thread Xuefu Zhang
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27627/#review60530 --- Ship it! Ship It! - Xuefu Zhang On Nov. 9, 2014, 10:39 p.m.,

Re: Review Request 27627: Split map-join plan into 2 SparkTasks in 3 stages [Spark Branch]

2014-11-08 Thread Xuefu Zhang
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27627/#review60482 ---

Re: Review Request 27627: Split map-join plan into 2 SparkTasks in 3 stages [Spark Branch]

2014-11-08 Thread Chao Sun
On Nov. 8, 2014, 3:15 p.m., Xuefu Zhang wrote: ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/SparkMapJoinResolver.java, line 214 https://reviews.apache.org/r/27627/diff/3/?file=754597#file754597line214 This assumes that result SparkWorks will be linearly dependent on

Re: Review Request 27627: Split map-join plan into 2 SparkTasks in 3 stages [Spark Branch]

2014-11-08 Thread Chao Sun
On Nov. 8, 2014, 12:44 a.m., Szehon Ho wrote: ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/SparkMapJoinResolver.java, line 224 https://reviews.apache.org/r/27627/diff/3/?file=754597#file754597line224 I've been thinking about this, as you had brought up a pretty rare

Re: Review Request 27627: Split map-join plan into 2 SparkTasks in 3 stages [Spark Branch]

2014-11-08 Thread Chao Sun
On Nov. 7, 2014, 11:07 p.m., Xuefu Zhang wrote: ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/SparkMapJoinResolver.java, line 100 https://reviews.apache.org/r/27627/diff/2/?file=754549#file754549line100 It seems possible that current is MJwork, right? Are you going to

Re: Review Request 27627: Split map-join plan into 2 SparkTasks in 3 stages [Spark Branch]

2014-11-08 Thread Xuefu Zhang
On Nov. 8, 2014, 3:15 p.m., Xuefu Zhang wrote: ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/SparkMapJoinResolver.java, line 214 https://reviews.apache.org/r/27627/diff/3/?file=754597#file754597line214 This assumes that result SparkWorks will be linearly dependent on

Re: Review Request 27627: Split map-join plan into 2 SparkTasks in 3 stages [Spark Branch]

2014-11-08 Thread Chao Sun
On Nov. 8, 2014, 3:15 p.m., Xuefu Zhang wrote: ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/SparkMapJoinResolver.java, line 214 https://reviews.apache.org/r/27627/diff/3/?file=754597#file754597line214 This assumes that result SparkWorks will be linearly dependent on

Re: Review Request 27627: Split map-join plan into 2 SparkTasks in 3 stages [Spark Branch]

2014-11-07 Thread Chao Sun
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27627/ --- (Updated Nov. 7, 2014, 3:57 p.m.) Review request for hive. Changes ---

Re: Review Request 27627: Split map-join plan into 2 SparkTasks in 3 stages [Spark Branch]

2014-11-07 Thread Chao Sun
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27627/ --- (Updated Nov. 7, 2014, 6:07 p.m.) Review request for hive. Changes ---

Re: Review Request 27627: Split map-join plan into 2 SparkTasks in 3 stages [Spark Branch]

2014-11-07 Thread Xuefu Zhang
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27627/#review60403 ---

Re: Review Request 27627: Split map-join plan into 2 SparkTasks in 3 stages [Spark Branch]

2014-11-07 Thread Szehon Ho
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27627/#review60380 ---

Review Request 27627: Split map-join plan into 2 SparkTasks in 3 stages [Spark Branch]

2014-11-05 Thread Chao Sun
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27627/ --- Review request for hive. Bugs: HIVE-8622

Re: Review Request 27627: Split map-join plan into 2 SparkTasks in 3 stages [Spark Branch]

2014-11-05 Thread Xuefu Zhang
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27627/#review59987 ---

Re: Review Request 27627: Split map-join plan into 2 SparkTasks in 3 stages [Spark Branch]

2014-11-05 Thread Szehon Ho
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27627/#review60034 --- Hi Chao, I left a review for a form of this patch at

Re: Review Request 27627: Split map-join plan into 2 SparkTasks in 3 stages [Spark Branch]

2014-11-05 Thread Chao Sun
On Nov. 5, 2014, 9:24 p.m., Szehon Ho wrote: Hi Chao, I left a review for a form of this patch at https://reviews.apache.org/r/27640/, as Suhas put it up for a separate review in combination with his patch. Thanks, I'll take a look there. - Chao

Re: Review Request 27627: Split map-join plan into 2 SparkTasks in 3 stages [Spark Branch]

2014-11-05 Thread Chao Sun
On Nov. 5, 2014, 7:16 p.m., Xuefu Zhang wrote: ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/SparkMapJoinResolver.java, line 128 https://reviews.apache.org/r/27627/diff/1/?file=750389#file750389line128 Do you mean parentTasks != null? That was a silly mistake. On