spark git commit: [SPARK-16240][ML] ML persistence backward compatibility for LDA - 2.0 backport

2016-09-22 Thread jkbradley
Repository: spark Updated Branches: refs/heads/branch-2.0 22216d6bd -> 54d4eee51 [SPARK-16240][ML] ML persistence backward compatibility for LDA - 2.0 backport ## What changes were proposed in this pull request? Allow Spark 2.x to load instances of LDA, LocalLDAModel, and

spark git commit: [SPARK-16719][ML] Random Forests should communicate fewer trees on each iteration

2016-09-22 Thread jkbradley
Repository: spark Updated Branches: refs/heads/master a4aeb7677 -> 947b8c6e3 [SPARK-16719][ML] Random Forests should communicate fewer trees on each iteration ## What changes were proposed in this pull request? RandomForest currently sends the entire forest to each worker on each

spark git commit: [SPARK-17639][BUILD] Add jce.jar to buildclasspath when building.

2016-09-22 Thread vanzin
Repository: spark Updated Branches: refs/heads/master 79159a1e8 -> a4aeb7677 [SPARK-17639][BUILD] Add jce.jar to buildclasspath when building. This was missing, preventing code that uses javax.crypto to properly compile in Spark. Author: Marcelo Vanzin Closes #15204

spark git commit: [SPARK-17502][17609][SQL][BACKPORT][2.0] Fix Multiple Bugs in DDL Statements on Temporary Views

2016-09-22 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.0 c393d86d1 -> 22216d6bd [SPARK-17502][17609][SQL][BACKPORT][2.0] Fix Multiple Bugs in DDL Statements on Temporary Views ### What changes were proposed in this pull request? This PR is to backport

[1/2] spark git commit: Preparing Spark release v2.0.1-rc2

2016-09-22 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-2.0 c2cb84165 -> c393d86d1 Preparing Spark release v2.0.1-rc2 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/04141ad4 Tree:

[2/2] spark git commit: Preparing development version 2.0.2-SNAPSHOT

2016-09-22 Thread pwendell
Preparing development version 2.0.2-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/c393d86d Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/c393d86d Diff:

[spark] Git Push Summary

2016-09-22 Thread pwendell
Repository: spark Updated Tags: refs/tags/v2.0.1-rc2 [created] 04141ad49 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

spark git commit: [SPARK-17635][SQL] Remove hardcode "agg_plan" in HashAggregateExec

2016-09-22 Thread rxin
Repository: spark Updated Branches: refs/heads/master a16619683 -> 79159a1e8 [SPARK-17635][SQL] Remove hardcode "agg_plan" in HashAggregateExec ## What changes were proposed in this pull request? "agg_plan" are hardcoded in HashAggregateExec, which have potential issue, so removing them.

spark git commit: [SPARK-17569][SPARK-17569][TEST] Make the unit test added for work again

2016-09-22 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master f4f6bd8c9 -> a16619683 [SPARK-17569][SPARK-17569][TEST] Make the unit test added for work again ## What changes were proposed in this pull request? A [PR](https://github.com/apache/spark/commit/a6aade0042d9c065669f46d2dac40ec6ce361e63)

spark git commit: [SPARK-16240][ML] ML persistence backward compatibility for LDA

2016-09-22 Thread jkbradley
Repository: spark Updated Branches: refs/heads/master 0d6348750 -> f4f6bd8c9 [SPARK-16240][ML] ML persistence backward compatibility for LDA ## What changes were proposed in this pull request? Allow Spark 2.x to load instances of LDA, LocalLDAModel, and DistributedLDAModel saved from Spark

spark git commit: [SPARK-17616][SQL] Support a single distinct aggregate combined with a non-partial aggregate

2016-09-22 Thread hvanhovell
Repository: spark Updated Branches: refs/heads/branch-2.0 47fc0b9f4 -> 0a593db36 [SPARK-17616][SQL] Support a single distinct aggregate combined with a non-partial aggregate We currently cannot execute an aggregate that contains a single distinct aggregate function and an one or more

spark git commit: [SPARK-17616][SQL] Support a single distinct aggregate combined with a non-partial aggregate

2016-09-22 Thread hvanhovell
Repository: spark Updated Branches: refs/heads/master 3cdae0ff2 -> 0d6348750 [SPARK-17616][SQL] Support a single distinct aggregate combined with a non-partial aggregate ## What changes were proposed in this pull request? We currently cannot execute an aggregate that contains a single

spark git commit: [SPARK-17638][STREAMING] Stop JVM StreamingContext when the Python process is dead

2016-09-22 Thread zsxwing
Repository: spark Updated Branches: refs/heads/branch-2.0 243bdb11d -> 47fc0b9f4 [SPARK-17638][STREAMING] Stop JVM StreamingContext when the Python process is dead ## What changes were proposed in this pull request? When the Python process is dead, the JVM StreamingContext is still running.

spark git commit: [SPARK-17638][STREAMING] Stop JVM StreamingContext when the Python process is dead

2016-09-22 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master 85d609cf2 -> 3cdae0ff2 [SPARK-17638][STREAMING] Stop JVM StreamingContext when the Python process is dead ## What changes were proposed in this pull request? When the Python process is dead, the JVM StreamingContext is still running.

spark git commit: [SPARK-17613] S3A base paths with no '/' at the end return empty DataFrames

2016-09-22 Thread joshrosen
Repository: spark Updated Branches: refs/heads/branch-2.0 f14f47f07 -> 243bdb11d [SPARK-17613] S3A base paths with no '/' at the end return empty DataFrames Consider you have a bucket as `s3a://some-bucket` and under it you have files: ``` s3a://some-bucket/file1.parquet

spark git commit: [SPARK-17613] S3A base paths with no '/' at the end return empty DataFrames

2016-09-22 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master 9f24a17c5 -> 85d609cf2 [SPARK-17613] S3A base paths with no '/' at the end return empty DataFrames ## What changes were proposed in this pull request? Consider you have a bucket as `s3a://some-bucket` and under it you have files: ```

spark git commit: Skip building R vignettes if Spark is not built

2016-09-22 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.0 b25a8e6e1 -> f14f47f07 Skip building R vignettes if Spark is not built ## What changes were proposed in this pull request? When we build the docs separately we don't have the JAR files from the Spark build in the same tree. As the

spark git commit: Skip building R vignettes if Spark is not built

2016-09-22 Thread rxin
Repository: spark Updated Branches: refs/heads/master 17b72d31e -> 9f24a17c5 Skip building R vignettes if Spark is not built ## What changes were proposed in this pull request? When we build the docs separately we don't have the JAR files from the Spark build in the same tree. As the SparkR

spark git commit: [SPARK-17485] Prevent failed remote reads of cached blocks from failing entire job (branch-1.6 backport)

2016-09-22 Thread joshrosen
Repository: spark Updated Branches: refs/heads/branch-1.6 ce0a222f5 -> 94524cef4 [SPARK-17485] Prevent failed remote reads of cached blocks from failing entire job (branch-1.6 backport) This patch is a branch-1.6 backport of #15037: ## What changes were proposed in this pull request? In

spark git commit: [SPARK-17365][CORE] Remove/Kill multiple executors together to reduce RPC call time.

2016-09-22 Thread vanzin
Repository: spark Updated Branches: refs/heads/master 8a02410a9 -> 17b72d31e [SPARK-17365][CORE] Remove/Kill multiple executors together to reduce RPC call time. ## What changes were proposed in this pull request? We are killing multiple executors together instead of iterating over expensive

spark git commit: [SQL][MINOR] correct the comment of SortBasedAggregationIterator.safeProj

2016-09-22 Thread lian
Repository: spark Updated Branches: refs/heads/master 72d9fba26 -> 8a02410a9 [SQL][MINOR] correct the comment of SortBasedAggregationIterator.safeProj ## What changes were proposed in this pull request? This comment went stale long time ago, this PR fixes it according to my understanding.

spark git commit: [SPARK-17281][ML][MLLIB] Add treeAggregateDepth parameter for AFTSurvivalRegression

2016-09-22 Thread yliang
Repository: spark Updated Branches: refs/heads/master 646f38346 -> 72d9fba26 [SPARK-17281][ML][MLLIB] Add treeAggregateDepth parameter for AFTSurvivalRegression ## What changes were proposed in this pull request? Add treeAggregateDepth parameter for AFTSurvivalRegression to keep consistent

spark git commit: [SPARK-17421][DOCS] Documenting the current treatment of MAVEN_OPTS.

2016-09-22 Thread srowen
Repository: spark Updated Branches: refs/heads/master de7df7def -> 646f38346 [SPARK-17421][DOCS] Documenting the current treatment of MAVEN_OPTS. ## What changes were proposed in this pull request? Modified the documentation to clarify that `build/mvn` and `pom.xml` always add Java

spark git commit: [SPARK-17421][DOCS] Documenting the current treatment of MAVEN_OPTS.

2016-09-22 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 e8b26be9b -> b25a8e6e1 [SPARK-17421][DOCS] Documenting the current treatment of MAVEN_OPTS. ## What changes were proposed in this pull request? Modified the documentation to clarify that `build/mvn` and `pom.xml` always add Java

spark git commit: [SPARK-17625][SQL] set expectedOutputAttributes when converting SimpleCatalogRelation to LogicalRelation

2016-09-22 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 3a80f92f8 -> de7df7def [SPARK-17625][SQL] set expectedOutputAttributes when converting SimpleCatalogRelation to LogicalRelation ## What changes were proposed in this pull request? We should set expectedOutputAttributes when converting