Re: Review Request 29787: HIVE-9257 : Merge spark to trunk January 2015 (Modified files)

2015-01-10 Thread Xuefu Zhang 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29787/#review67603
---



pom.xml


A followup to get rid of this?



ql/src/java/org/apache/hadoop/hive/ql/exec/ExplainTask.java


We should restrain from using * in imports.


- Xuefu Zhang


On Jan. 9, 2015, 11:55 p.m., Szehon Ho wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/29787/
> ---
> 
> (Updated Jan. 9, 2015, 11:55 p.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-9257
> https://issues.apache.org/jira/browse/HIVE-9257
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> As the entire patch is too big, this shows the modified files.  These have 
> been cleanuped as part of HIVE-9319, HIVE-9306, HIVE-9305.
> 
> The new files can be found here:  
> http://svn.apache.org/repos/asf/hive/branches/spark/ or 
> https://github.com/apache/hive/tree/spark under:
> # data/conf/spark/
> # 
> itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcWithLocalClusterSpark.java
> # 
> itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestMultiSessionsHS2WithLocalClusterSpark.java
> # itests/qtest-spark/
> # 
> ql/src/java/org/apache/hadoop/hive/ql/exec/SparkHashTableSinkOperator.java
> # ql/src/java/org/apache/hadoop/hive/ql/exec/spark/
> # ql/src/java/org/apache/hadoop/hive/ql/lib/TypeRule.java
> # 
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/SparkMapJoinProcessor.java
> # 
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/GenSparkSkewJoinProcessor.java
> # 
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/SparkCrossProductCheck.java
> # 
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/SparkMapJoinResolver.java
> # ql/src/java/org/apache/hadoop/hive/ql/optimizer/spark/
> # ql/src/java/org/apache/hadoop/hive/ql/parse/spark/
> # 
> ql/src/java/org/apache/hadoop/hive/ql/plan/SparkBucketMapJoinContext.java
> # ql/src/java/org/apache/hadoop/hive/ql/plan/SparkEdgeProperty.java
> # ql/src/java/org/apache/hadoop/hive/ql/plan/SparkHashTableSinkDesc.java
> # ql/src/java/org/apache/hadoop/hive/ql/plan/SparkWork.java
> # 
> ql/src/java/org/apache/hadoop/hive/ql/stats/CounterStatsAggregatorSpark.java
> # ql/src/test/org/apache/hadoop/hive/ql/exec/spark/
> # ql/src/test/queries/clientpositive/auto_join_stats.q
> # ql/src/test/queries/clientpositive/auto_join_stats2.q
> # ql/src/test/queries/clientpositive/bucket_map_join_spark1.q
> # ql/src/test/queries/clientpositive/bucket_map_join_spark2.q
> # ql/src/test/queries/clientpositive/bucket_map_join_spark3.q
> # ql/src/test/queries/clientpositive/bucket_map_join_spark4.q
> # ql/src/test/queries/clientpositive/multi_insert_mixed.q
> # ql/src/test/queries/clientpositive/multi_insert_union_src.q
> # ql/src/test/queries/clientpositive/parallel_join0.q
> # ql/src/test/queries/clientpositive/parallel_join1.q
> # ql/src/test/queries/clientpositive/spark_test.q
> # ql/src/test/queries/clientpositive/udf_example_add.q
> # ql/src/test/results/clientpositive/auto_join_stats.q.out
> # ql/src/test/results/clientpositive/auto_join_stats2.q.out
> # ql/src/test/results/clientpositive/bucket_map_join_spark1.q.out
> # ql/src/test/results/clientpositive/bucket_map_join_spark2.q.out
> # ql/src/test/results/clientpositive/bucket_map_join_spark3.q.out
> # ql/src/test/results/clientpositive/bucket_map_join_spark4.q.out
> # ql/src/test/results/clientpositive/multi_insert_mixed.q.out
> # ql/src/test/results/clientpositive/multi_insert_union_src.q.out
> # ql/src/test/results/clientpositive/parallel_join0.q.out
> # ql/src/test/results/clientpositive/parallel_join1.q.out
> # ql/src/test/results/clientpositive/spark/
> # ql/src/test/results/clientpositive/spark_test.q.out
> # ql/src/test/results/clientpositive/udf_example_add.q.out
> # spark-client/
> Cleanup and review of those have been done as part of HIVE-9281 and HIVE-9288.
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/common/StatsSetupConst.java cd4beeb 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 8264b16 
>   data/conf/hive-log4j.properties a5b9c9a 
>   itests/hive-unit/pom.xml f9f59c9 
>   itests/pom.xml 0a154d6 
>   itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java 878202a 
>   pom.xml efe5e3a 
>   ql/pom.xml 84e912e 
>   ql/src/java/org/apache/hadoop/hive/ql/Context.java 0373273 
>   ql/src/java/org/apache/hadoop/hive/ql/Driver.java 8bb6d0f 
>   ql/src/java/org/apache/had

Review Request 29787: HIVE-9257 : Merge spark to trunk January 2015 (Modified files)

2015-01-09 Thread Szehon Ho 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29787/
---

Review request for hive.


Bugs: HIVE-9257
https://issues.apache.org/jira/browse/HIVE-9257


Repository: hive-git


Description
---

As the entire patch is too big, this shows the modified files.  These have been 
cleanuped as part of HIVE-9319, HIVE-9306, HIVE-9305.

The new files can be found here:  
http://svn.apache.org/repos/asf/hive/branches/spark/ or 
https://github.com/apache/hive/tree/spark under:
#   data/conf/spark/
#   
itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcWithLocalClusterSpark.java
#   
itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestMultiSessionsHS2WithLocalClusterSpark.java
#   itests/qtest-spark/
#   
ql/src/java/org/apache/hadoop/hive/ql/exec/SparkHashTableSinkOperator.java
#   ql/src/java/org/apache/hadoop/hive/ql/exec/spark/
#   ql/src/java/org/apache/hadoop/hive/ql/lib/TypeRule.java
#   
ql/src/java/org/apache/hadoop/hive/ql/optimizer/SparkMapJoinProcessor.java
#   
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/GenSparkSkewJoinProcessor.java
#   
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/SparkCrossProductCheck.java
#   
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/SparkMapJoinResolver.java
#   ql/src/java/org/apache/hadoop/hive/ql/optimizer/spark/
#   ql/src/java/org/apache/hadoop/hive/ql/parse/spark/
#   
ql/src/java/org/apache/hadoop/hive/ql/plan/SparkBucketMapJoinContext.java
#   ql/src/java/org/apache/hadoop/hive/ql/plan/SparkEdgeProperty.java
#   ql/src/java/org/apache/hadoop/hive/ql/plan/SparkHashTableSinkDesc.java
#   ql/src/java/org/apache/hadoop/hive/ql/plan/SparkWork.java
#   
ql/src/java/org/apache/hadoop/hive/ql/stats/CounterStatsAggregatorSpark.java
#   ql/src/test/org/apache/hadoop/hive/ql/exec/spark/
#   ql/src/test/queries/clientpositive/auto_join_stats.q
#   ql/src/test/queries/clientpositive/auto_join_stats2.q
#   ql/src/test/queries/clientpositive/bucket_map_join_spark1.q
#   ql/src/test/queries/clientpositive/bucket_map_join_spark2.q
#   ql/src/test/queries/clientpositive/bucket_map_join_spark3.q
#   ql/src/test/queries/clientpositive/bucket_map_join_spark4.q
#   ql/src/test/queries/clientpositive/multi_insert_mixed.q
#   ql/src/test/queries/clientpositive/multi_insert_union_src.q
#   ql/src/test/queries/clientpositive/parallel_join0.q
#   ql/src/test/queries/clientpositive/parallel_join1.q
#   ql/src/test/queries/clientpositive/spark_test.q
#   ql/src/test/queries/clientpositive/udf_example_add.q
#   ql/src/test/results/clientpositive/auto_join_stats.q.out
#   ql/src/test/results/clientpositive/auto_join_stats2.q.out
#   ql/src/test/results/clientpositive/bucket_map_join_spark1.q.out
#   ql/src/test/results/clientpositive/bucket_map_join_spark2.q.out
#   ql/src/test/results/clientpositive/bucket_map_join_spark3.q.out
#   ql/src/test/results/clientpositive/bucket_map_join_spark4.q.out
#   ql/src/test/results/clientpositive/multi_insert_mixed.q.out
#   ql/src/test/results/clientpositive/multi_insert_union_src.q.out
#   ql/src/test/results/clientpositive/parallel_join0.q.out
#   ql/src/test/results/clientpositive/parallel_join1.q.out
#   ql/src/test/results/clientpositive/spark/
#   ql/src/test/results/clientpositive/spark_test.q.out
#   ql/src/test/results/clientpositive/udf_example_add.q.out
#   spark-client/
Cleanup and review of those have been done as part of HIVE-9281 and HIVE-9288.


Diffs
-

  common/src/java/org/apache/hadoop/hive/common/StatsSetupConst.java cd4beeb 
  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 8264b16 
  data/conf/hive-log4j.properties a5b9c9a 
  itests/hive-unit/pom.xml f9f59c9 
  itests/pom.xml 0a154d6 
  itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java 878202a 
  pom.xml efe5e3a 
  ql/pom.xml 84e912e 
  ql/src/java/org/apache/hadoop/hive/ql/Context.java 0373273 
  ql/src/java/org/apache/hadoop/hive/ql/Driver.java 8bb6d0f 
  ql/src/java/org/apache/hadoop/hive/ql/HashTableLoaderFactory.java 10ad933 
  ql/src/java/org/apache/hadoop/hive/ql/exec/ExplainTask.java 3f5f475 
  ql/src/java/org/apache/hadoop/hive/ql/exec/FilterOperator.java b96faa4 
  ql/src/java/org/apache/hadoop/hive/ql/exec/HashTableSinkOperator.java f0e04e7 
  ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorFactory.java 561bf3b 
  ql/src/java/org/apache/hadoop/hive/ql/exec/ScriptOperator.java 87eaa8f 
  ql/src/java/org/apache/hadoop/hive/ql/exec/TaskFactory.java e3469a0 
  ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java ed03bb9 
  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecMapperContext.java 8b92f32 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/persistence