[2/2] spark git commit: [SPARK-19464][CORE][YARN][TEST-HADOOP2.6] Remove support for Hadoop 2.5 and earlier

2017-02-08 Thread srowen
[SPARK-19464][CORE][YARN][TEST-HADOOP2.6] Remove support for Hadoop 2.5 and earlier ## What changes were proposed in this pull request? - Remove support for Hadoop 2.5 and earlier - Remove reflection and code constructs only needed to support multiple versions at once - Update docs to reflect n

[1/2] spark git commit: [SPARK-19464][CORE][YARN][TEST-HADOOP2.6] Remove support for Hadoop 2.5 and earlier

2017-02-08 Thread srowen
Repository: spark Updated Branches: refs/heads/master d60dde26f -> e8d3fca45 http://git-wip-us.apache.org/repos/asf/spark/blob/e8d3fca4/resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ClientSuite.scala -- dif

spark git commit: [SPARK-19409][BUILD][TEST-MAVEN] Fix ParquetAvroCompatibilitySuite failure due to test dependency on avro

2017-02-08 Thread srowen
Repository: spark Updated Branches: refs/heads/master e8d3fca45 -> 0077bfcb9 [SPARK-19409][BUILD][TEST-MAVEN] Fix ParquetAvroCompatibilitySuite failure due to test dependency on avro ## What changes were proposed in this pull request? After using Apache Parquet 1.8.2, `ParquetAvroCompatibili

spark git commit: [SPARK-19279][SQL][FOLLOW-UP] Infer Schema for Hive Serde Tables

2017-02-08 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 0077bfcb9 -> 4d4d0de7f [SPARK-19279][SQL][FOLLOW-UP] Infer Schema for Hive Serde Tables ### What changes were proposed in this pull request? `table.schema` is always not empty for partitioned tables, because `table.schema` also contains th

spark git commit: [SPARK-19464][BUILD][HOTFIX][TEST-HADOOP2.6] Add back mockito test dep in YARN module, as it ends up being required in a Maven build

2017-02-08 Thread srowen
Repository: spark Updated Branches: refs/heads/master 4d4d0de7f -> 15627ac74 [SPARK-19464][BUILD][HOTFIX][TEST-HADOOP2.6] Add back mockito test dep in YARN module, as it ends up being required in a Maven build Add back mockito test dep in YARN module, as it ends up being required in a Maven

spark git commit: [SPARK-19400][ML] Allow GLM to handle intercept only model

2017-02-08 Thread jkbradley
Repository: spark Updated Branches: refs/heads/master 15627ac74 -> 1aeb9f6cb [SPARK-19400][ML] Allow GLM to handle intercept only model ## What changes were proposed in this pull request? Intercept-only GLM is failing for non-Gaussian family because of reducing an empty array in IWLS. The fol

spark git commit: [SPARK-19413][SS] MapGroupsWithState for arbitrary stateful operations for branch-2.1

2017-02-08 Thread zsxwing
Repository: spark Updated Branches: refs/heads/branch-2.1 71b6eacf7 -> 502c927b8 [SPARK-19413][SS] MapGroupsWithState for arbitrary stateful operations for branch-2.1 This is a follow up PR for merging #16758 to spark 2.1 branch ## What changes were proposed in this pull request? `mapGroups

spark git commit: [SPARK-19464][BUILD][HOTFIX] run-tests should use hadoop2.6

2017-02-08 Thread srowen
Repository: spark Updated Branches: refs/heads/master 1aeb9f6cb -> c618ccdbe [SPARK-19464][BUILD][HOTFIX] run-tests should use hadoop2.6 ## What changes were proposed in this pull request? After SPARK-19464, **SparkPullRequestBuilder** fails because it still tries to use hadoop2.3. **BEFORE

spark git commit: [SPARK-18872][SQL][TESTS] New test cases for EXISTS subquery (Aggregate, Having, Orderby, Limit)

2017-02-08 Thread lixiao
Repository: spark Updated Branches: refs/heads/master c618ccdbe -> 64cae22f7 [SPARK-18872][SQL][TESTS] New test cases for EXISTS subquery (Aggregate, Having, Orderby, Limit) ## What changes were proposed in this pull request? This PR adds the second set of tests for EXISTS subquery. File nam

spark git commit: [SPARK-19359][SQL] renaming partition should not leave useless directories

2017-02-08 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 64cae22f7 -> 50a991264 [SPARK-19359][SQL] renaming partition should not leave useless directories ## What changes were proposed in this pull request? Hive metastore is not case-preserving and keep partition columns with lower case names.

spark git commit: [SPARK-19265][SQL][FOLLOW-UP] Configurable `tableRelationCache` maximum size

2017-02-08 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 50a991264 -> 9d9d67c79 [SPARK-19265][SQL][FOLLOW-UP] Configurable `tableRelationCache` maximum size ## What changes were proposed in this pull request? SPARK-19265 had made table relation cache general; this follow-up aims to make `tableR