Re: Review Request 34666: HIVE-9152 - Dynamic Partition Pruning [Spark Branch]

2015-07-12 Thread chengxiang li
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34666/#review91427 --- Ship it! Ship It! - chengxiang li On 七月 8, 2015, 6:04 p.m.,

Re: Review Request 34666: HIVE-9152 - Dynamic Partition Pruning [Spark Branch]

2015-07-08 Thread Chao Sun
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34666/ --- (Updated July 8, 2015, 6:04 p.m.) Review request for hive, chengxiang li and

Re: Review Request 34666: HIVE-9152 - Dynamic Partition Pruning [Spark Branch]

2015-07-07 Thread Chao Sun
On June 30, 2015, 8:55 p.m., Xuefu Zhang wrote: ql/src/java/org/apache/hadoop/hive/ql/parse/spark/SparkPartitionPruningSinkOperator.java, line 92 https://reviews.apache.org/r/34666/diff/1/?file=971715#file971715line92 Can we still get conflicts in the file name? Chao Sun

Re: Review Request 34666: HIVE-9152 - Dynamic Partition Pruning [Spark Branch]

2015-07-07 Thread Chao Sun
On July 6, 2015, 10:26 p.m., Xuefu Zhang wrote: ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorSparkPartitionPruningSinkOperator.java, line 77 https://reviews.apache.org/r/34666/diff/2/?file=999023#file999023line77 I guess I don't know enough to comment on this, but

Re: Review Request 34666: HIVE-9152 - Dynamic Partition Pruning [Spark Branch]

2015-07-07 Thread Chao Sun
On May 27, 2015, 6:52 p.m., Xuefu Zhang wrote: ql/if/queryplan.thrift, line 60 https://reviews.apache.org/r/34666/diff/1/?file=971689#file971689line60 I'm not sure if it matters, but it's probably better if we add it as the last. Xuefu Zhang wrote: It's still needed to

Re: Review Request 34666: HIVE-9152 - Dynamic Partition Pruning [Spark Branch]

2015-07-07 Thread Chao Sun
On July 2, 2015, 6:36 a.m., chengxiang li wrote: ql/src/java/org/apache/hadoop/hive/ql/optimizer/SparkRemoveDynamicPruningBySize.java, line 59 https://reviews.apache.org/r/34666/diff/1/?file=971706#file971706line59 The statistic data shoud be quite unaccurate after filter and

Re: Review Request 34666: HIVE-9152 - Dynamic Partition Pruning [Spark Branch]

2015-07-07 Thread Chao Sun
On June 30, 2015, 8:55 p.m., Xuefu Zhang wrote: ql/src/java/org/apache/hadoop/hive/ql/parse/spark/SparkPartitionPruningSinkOperator.java, line 92 https://reviews.apache.org/r/34666/diff/1/?file=971715#file971715line92 Can we still get conflicts in the file name? Chao Sun

Re: Review Request 34666: HIVE-9152 - Dynamic Partition Pruning [Spark Branch]

2015-07-06 Thread chengxiang li
On 七月 2, 2015, 6:36 a.m., chengxiang li wrote: ql/src/java/org/apache/hadoop/hive/ql/optimizer/SparkRemoveDynamicPruningBySize.java, line 59 https://reviews.apache.org/r/34666/diff/1/?file=971706#file971706line59 The statistic data shoud be quite unaccurate after filter and

Re: Review Request 34666: HIVE-9152 - Dynamic Partition Pruning [Spark Branch]

2015-07-06 Thread Xuefu Zhang
On May 27, 2015, 6:52 p.m., Xuefu Zhang wrote: ql/if/queryplan.thrift, line 60 https://reviews.apache.org/r/34666/diff/1/?file=971689#file971689line60 I'm not sure if it matters, but it's probably better if we add it as the last. It's still needed to move to the last as other

Re: Review Request 34666: HIVE-9152 - Dynamic Partition Pruning [Spark Branch]

2015-07-06 Thread Gopal V
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34666/#review90598 --- ql/if/queryplan.thrift (line 60)

Re: Review Request 34666: HIVE-9152 - Dynamic Partition Pruning [Spark Branch]

2015-07-06 Thread Xuefu Zhang
On May 27, 2015, 6:52 p.m., Xuefu Zhang wrote: ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkUtilities.java, line 177 https://reviews.apache.org/r/34666/diff/1/?file=971700#file971700line177 Any chance that an op might be visited multiple times? Chao Sun wrote: It

Re: Review Request 34666: HIVE-9152 - Dynamic Partition Pruning [Spark Branch]

2015-07-06 Thread Xuefu Zhang
On June 30, 2015, 8:55 p.m., Xuefu Zhang wrote: ql/src/java/org/apache/hadoop/hive/ql/parse/spark/SparkPartitionPruningSinkOperator.java, line 92 https://reviews.apache.org/r/34666/diff/1/?file=971715#file971715line92 Can we still get conflicts in the file name? Chao Sun

Re: Review Request 34666: HIVE-9152 - Dynamic Partition Pruning [Spark Branch]

2015-07-06 Thread Xuefu Zhang
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34666/#review90593 ---

Re: Review Request 34666: HIVE-9152 - Dynamic Partition Pruning [Spark Branch]

2015-07-03 Thread Chao Sun
On June 30, 2015, 8:55 p.m., Xuefu Zhang wrote: itests/src/test/resources/testconfiguration.properties, line 894 https://reviews.apache.org/r/34666/diff/1/?file=971683#file971683line894 Are there more test cases that can be turned on? will turn on

Re: Review Request 34666: HIVE-9152 - Dynamic Partition Pruning [Spark Branch]

2015-07-03 Thread Chao Sun
On July 1, 2015, midnight, Xuefu Zhang wrote: ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/SparkMapJoinResolver.java, line 142 https://reviews.apache.org/r/34666/diff/1/?file=971707#file971707line142 Why do we need this now? This is to prevent a newly generated task

Re: Review Request 34666: HIVE-9152 - Dynamic Partition Pruning [Spark Branch]

2015-07-03 Thread Chao Sun
On July 2, 2015, 6:36 a.m., chengxiang li wrote: ql/src/java/org/apache/hadoop/hive/ql/optimizer/SparkRemoveDynamicPruningBySize.java, line 59 https://reviews.apache.org/r/34666/diff/1/?file=971706#file971706line59 The statistic data shoud be quite unaccurate after filter and

Re: Review Request 34666: HIVE-9152 - Dynamic Partition Pruning [Spark Branch]

2015-07-03 Thread Chao Sun
On July 2, 2015, 5:25 a.m., chengxiang li wrote: ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkDynamicPartitionPruner.java, line 251 https://reviews.apache.org/r/34666/diff/1/?file=971699#file971699line251 Log in error level should means some error happens,the process

Re: Review Request 34666: HIVE-9152 - Dynamic Partition Pruning [Spark Branch]

2015-07-03 Thread Chao Sun
On May 27, 2015, 6:52 p.m., Xuefu Zhang wrote: ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkUtilities.java, line 177 https://reviews.apache.org/r/34666/diff/1/?file=971700#file971700line177 Any chance that an op might be visited multiple times? It shouldn't - it'a tree

Re: Review Request 34666: HIVE-9152 - Dynamic Partition Pruning [Spark Branch]

2015-07-03 Thread Chao Sun
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34666/ --- (Updated July 3, 2015, 10:45 p.m.) Review request for hive, chengxiang li and

Re: Review Request 34666: HIVE-9152 - Dynamic Partition Pruning [Spark Branch]

2015-07-02 Thread chengxiang li
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34666/#review90197 ---

Re: Review Request 34666: HIVE-9152 - Dynamic Partition Pruning [Spark Branch]

2015-07-01 Thread chengxiang li
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34666/#review90191 ---

Re: Review Request 34666: HIVE-9152 - Dynamic Partition Pruning [Spark Branch]

2015-06-30 Thread Xuefu Zhang
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34666/#review89972 ---

Re: Review Request 34666: HIVE-9152 - Dynamic Partition Pruning [Spark Branch]

2015-06-30 Thread Xuefu Zhang
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34666/#review89826 --- itests/src/test/resources/testconfiguration.properties (line 894)

Re: Review Request 34666: HIVE-9152 - Dynamic Partition Pruning [Spark Branch]

2015-05-27 Thread Xuefu Zhang
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34666/#review85230 --- This a big patch, for a big feature. It's hard to review offline.