spark git commit: [SQL][DOC] updating doc for JSON source to link to jsonlines.org

2016-10-26 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master 1dbe9896b -> 44c8bfda7 [SQL][DOC] updating doc for JSON source to link to jsonlines.org ## What changes were proposed in this pull request? API and programming guide doc changes for Scala, Python and R. ## How was this patch tested? manu

spark git commit: [SPARK-17157][SPARKR][FOLLOW-UP] doc fixes

2016-10-26 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master d3b4831d0 -> 1dbe9896b [SPARK-17157][SPARKR][FOLLOW-UP] doc fixes ## What changes were proposed in this pull request? a couple of small late finding fixes for doc ## How was this patch tested? manually wangmiao1981 Author: Felix Cheung

spark git commit: [SPARK-18132] Fix checkstyle

2016-10-26 Thread yhuai
Repository: spark Updated Branches: refs/heads/branch-2.0 dcf2f090c -> 1a4be51d6 [SPARK-18132] Fix checkstyle This PR fixes checkstyle. Author: Yin Huai Closes #15656 from yhuai/fix-format. (cherry picked from commit d3b4831d009905185ad74096ce3ecfa934bc191d) Signed-off-by: Yin Huai Proj

spark git commit: [SPARK-18132] Fix checkstyle

2016-10-26 Thread yhuai
Repository: spark Updated Branches: refs/heads/master dd4f088c1 -> d3b4831d0 [SPARK-18132] Fix checkstyle This PR fixes checkstyle. Author: Yin Huai Closes #15656 from yhuai/fix-format. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/a

spark git commit: [SPARK-18009][SQL] Fix ClassCastException while calling toLocalIterator() on dataframe produced by RunnableCommand

2016-10-26 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.0 ea205e376 -> dcf2f090c [SPARK-18009][SQL] Fix ClassCastException while calling toLocalIterator() on dataframe produced by RunnableCommand A short code snippet that uses toLocalIterator() on a dataframe produced by a RunnableCommand re

spark git commit: [SPARK-18009][SQL] Fix ClassCastException while calling toLocalIterator() on dataframe produced by RunnableCommand

2016-10-26 Thread wenchen
Repository: spark Updated Branches: refs/heads/master f1aeed8b0 -> dd4f088c1 [SPARK-18009][SQL] Fix ClassCastException while calling toLocalIterator() on dataframe produced by RunnableCommand ## What changes were proposed in this pull request? A short code snippet that uses toLocalIterator()

spark git commit: [SPARK-17770][CATALYST] making ObjectType public

2016-10-26 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master 5b27598ff -> f1aeed8b0 [SPARK-17770][CATALYST] making ObjectType public ## What changes were proposed in this pull request? In order to facilitate the writing of additional Encoders, I proposed opening up the ObjectType SQL DataType. This

spark git commit: [SPARK-16963][STREAMING][SQL] Changes to Source trait and related implementation classes

2016-10-26 Thread zsxwing
Repository: spark Updated Branches: refs/heads/branch-2.0 72b3cff33 -> ea205e376 [SPARK-16963][STREAMING][SQL] Changes to Source trait and related implementation classes ## What changes were proposed in this pull request? This PR contains changes to the Source trait such that the scheduler c

spark git commit: [SPARK-16963][STREAMING][SQL] Changes to Source trait and related implementation classes

2016-10-26 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master a76846cfb -> 5b27598ff [SPARK-16963][STREAMING][SQL] Changes to Source trait and related implementation classes ## What changes were proposed in this pull request? This PR contains changes to the Source trait such that the scheduler can n

spark git commit: [SPARK-18126][SPARK-CORE] getIteratorZipWithIndex accepts negative value as index

2016-10-26 Thread rxin
Repository: spark Updated Branches: refs/heads/master 29cea8f33 -> a76846cfb [SPARK-18126][SPARK-CORE] getIteratorZipWithIndex accepts negative value as index ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) `Utils.getIteratorZipWithIndex` wa

spark git commit: [SPARK-17157][SPARKR] Add multiclass logistic regression SparkR Wrapper

2016-10-26 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master 5b7d403c1 -> 29cea8f33 [SPARK-17157][SPARKR] Add multiclass logistic regression SparkR Wrapper ## What changes were proposed in this pull request? As we discussed in #14818, I added a separate R wrapper spark.logit for logistic regression

spark git commit: [SPARK-18094][SQL][TESTS] Move group analytics test cases from `SQLQuerySuite` into a query file test.

2016-10-26 Thread rxin
Repository: spark Updated Branches: refs/heads/master dcdda1978 -> 5b7d403c1 [SPARK-18094][SQL][TESTS] Move group analytics test cases from `SQLQuerySuite` into a query file test. ## What changes were proposed in this pull request? Currently we have several test cases for group analytics(ROL

[2/2] spark git commit: Preparing development version 2.0.3-SNAPSHOT

2016-10-26 Thread pwendell
Preparing development version 2.0.3-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/72b3cff3 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/72b3cff3 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/72b

[1/2] spark git commit: Preparing Spark release v2.0.2-rc1

2016-10-26 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-2.0 76b71eef4 -> 72b3cff33 Preparing Spark release v2.0.2-rc1 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/1c2908ee Tree: http://git-wip-us.apache.org/repos/asf/spar

[spark] Git Push Summary

2016-10-26 Thread pwendell
Repository: spark Updated Tags: refs/tags/v2.0.2-rc1 [created] 1c2908eeb - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

spark git commit: [SPARK-14300][DOCS][MLLIB] Scala MLlib examples code merge and clean up

2016-10-26 Thread jkbradley
Repository: spark Updated Branches: refs/heads/master fb0a8a8dd -> dcdda1978 [SPARK-14300][DOCS][MLLIB] Scala MLlib examples code merge and clean up ## What changes were proposed in this pull request? https://issues.apache.org/jira/browse/SPARK-14300 Duplicated code found in scala/examples/m

spark git commit: [SPARK-17961][SPARKR][SQL] Add storageLevel to DataFrame for SparkR

2016-10-26 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master ea3605e82 -> fb0a8a8dd [SPARK-17961][SPARKR][SQL] Add storageLevel to DataFrame for SparkR ## What changes were proposed in this pull request? Add storageLevel to DataFrame for SparkR. This is similar to this RP: https://github.com/apache

spark git commit: [SPARK-13747][SQL] Fix concurrent executions in ForkJoinPool for SQL (branch 2.0)

2016-10-26 Thread zsxwing
Repository: spark Updated Branches: refs/heads/branch-2.0 b482b3d58 -> 76b71eef4 [SPARK-13747][SQL] Fix concurrent executions in ForkJoinPool for SQL (branch 2.0) ## What changes were proposed in this pull request? Backport #15520 to 2.0. ## How was this patch tested? Jenkins Author: Shix

spark git commit: [MINOR][ML] Refactor clustering summary.

2016-10-26 Thread jkbradley
Repository: spark Updated Branches: refs/heads/master 7d10631c1 -> ea3605e82 [MINOR][ML] Refactor clustering summary. ## What changes were proposed in this pull request? Abstract ```ClusteringSummary``` from ```KMeansSummary```, ```GaussianMixtureSummary``` and ```BisectingSummary```, and eli

spark git commit: [SPARK-18104][DOC] Don't build KafkaSource doc

2016-10-26 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master fa7d9d708 -> 7d10631c1 [SPARK-18104][DOC] Don't build KafkaSource doc ## What changes were proposed in this pull request? Don't need to build doc for KafkaSource because the user should use the data source APIs to use KafkaSource. All Kaf

spark git commit: [SPARK-18104][DOC] Don't build KafkaSource doc

2016-10-26 Thread zsxwing
Repository: spark Updated Branches: refs/heads/branch-2.0 5b81b0102 -> b482b3d58 [SPARK-18104][DOC] Don't build KafkaSource doc ## What changes were proposed in this pull request? Don't need to build doc for KafkaSource because the user should use the data source APIs to use KafkaSource. All

spark git commit: [SPARK-18063][SQL] Failed to infer constraints over multiple aliases

2016-10-26 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.0 773fbfef1 -> 5b81b0102 [SPARK-18063][SQL] Failed to infer constraints over multiple aliases ## What changes were proposed in this pull request? The `UnaryNode.getAliasedConstraints` function fails to replace all expressions by their a

spark git commit: [SPARK-18063][SQL] Failed to infer constraints over multiple aliases

2016-10-26 Thread rxin
Repository: spark Updated Branches: refs/heads/master 7ac70e7ba -> fa7d9d708 [SPARK-18063][SQL] Failed to infer constraints over multiple aliases ## What changes were proposed in this pull request? The `UnaryNode.getAliasedConstraints` function fails to replace all expressions by their alias

spark git commit: [SPARK-13747][SQL] Fix concurrent executions in ForkJoinPool for SQL

2016-10-26 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master 312ea3f7f -> 7ac70e7ba [SPARK-13747][SQL] Fix concurrent executions in ForkJoinPool for SQL ## What changes were proposed in this pull request? Calling `Await.result` will allow other tasks to be run on the same thread when using ForkJoin

spark git commit: [SPARK-16304] LinkageError should not crash Spark executor

2016-10-26 Thread zsxwing
Repository: spark Updated Branches: refs/heads/branch-2.0 b4a7b6551 -> 773fbfef1 [SPARK-16304] LinkageError should not crash Spark executor ## What changes were proposed in this pull request? This patch updates the failure handling logic so Spark executor does not crash when seeing LinkageErr

spark git commit: [SPARK-17748][FOLLOW-UP][ML] Reorg variables of WeightedLeastSquares.

2016-10-26 Thread yliang
Repository: spark Updated Branches: refs/heads/master 4bee95407 -> 312ea3f7f [SPARK-17748][FOLLOW-UP][ML] Reorg variables of WeightedLeastSquares. ## What changes were proposed in this pull request? This is follow-up work of #15394. Reorg some variables of ```WeightedLeastSquares``` and fix on

spark git commit: [SPARK-18093][SQL] Fix default value test in SQLConfSuite to work rega…

2016-10-26 Thread vanzin
Repository: spark Updated Branches: refs/heads/branch-2.0 192c1dd4f -> b4a7b6551 [SPARK-18093][SQL] Fix default value test in SQLConfSuite to work rega… …rdless of warehouse dir's existence ## What changes were proposed in this pull request? Appending a trailing slash, if there already is

spark git commit: [SPARK-18093][SQL] Fix default value test in SQLConfSuite to work rega…

2016-10-26 Thread vanzin
Repository: spark Updated Branches: refs/heads/master 3c023570b -> 4bee95407 [SPARK-18093][SQL] Fix default value test in SQLConfSuite to work rega… …rdless of warehouse dir's existence ## What changes were proposed in this pull request? Appending a trailing slash, if there already isn't

svn commit: r16706 - in /release/spark: spark-1.5.2/ spark-1.6.1/ spark-2.0.0-preview/ spark-2.0.0/

2016-10-26 Thread srowen
Author: srowen Date: Wed Oct 26 15:25:03 2016 New Revision: 16706 Log: Remove non-current Spark releases from dist Removed: release/spark/spark-1.5.2/ release/spark/spark-1.6.1/ release/spark/spark-2.0.0/ release/spark/spark-2.0.0-preview/ ---

spark git commit: [SPARK-17733][SQL] InferFiltersFromConstraints rule never terminates for query

2016-10-26 Thread hvanhovell
Repository: spark Updated Branches: refs/heads/branch-2.0 c2cce2e60 -> 192c1dd4f [SPARK-17733][SQL] InferFiltersFromConstraints rule never terminates for query ## What changes were proposed in this pull request? The function `QueryPlan.inferAdditionalConstraints` and `UnaryNode.getAliasedCon

spark git commit: [SPARK-17733][SQL] InferFiltersFromConstraints rule never terminates for query

2016-10-26 Thread hvanhovell
Repository: spark Updated Branches: refs/heads/master 402205ddf -> 3c023570b [SPARK-17733][SQL] InferFiltersFromConstraints rule never terminates for query ## What changes were proposed in this pull request? The function `QueryPlan.inferAdditionalConstraints` and `UnaryNode.getAliasedConstra

spark-website git commit: Link to archive.apache.org for Spark releases besides 2.0.1 and 1.6.2; comment/remove some dead JS in this file

2016-10-26 Thread srowen
Repository: spark-website Updated Branches: refs/heads/asf-site 4150a7329 -> f284a2687 Link to archive.apache.org for Spark releases besides 2.0.1 and 1.6.2; comment/remove some dead JS in this file Project: http://git-wip-us.apache.org/repos/asf/spark-website/repo Commit: http://git-wip-us.

spark git commit: [SPARK-17802] Improved caller context logging.

2016-10-26 Thread srowen
Repository: spark Updated Branches: refs/heads/master 5d0f81da4 -> 402205ddf [SPARK-17802] Improved caller context logging. ## What changes were proposed in this pull request? [SPARK-16757](https://issues.apache.org/jira/browse/SPARK-16757) sets the hadoop `CallerContext` when calling hadoop

spark git commit: [SPARK-4411][WEB UI] Add "kill" link for jobs in the UI

2016-10-26 Thread srowen
Repository: spark Updated Branches: refs/heads/master 297813647 -> 5d0f81da4 [SPARK-4411][WEB UI] Add "kill" link for jobs in the UI ## What changes were proposed in this pull request? Currently users can kill stages via the web ui but not jobs directly (jobs are killed if one of their stage

spark git commit: [SPARK-18027][YARN] .sparkStaging not clean on RM ApplicationNotFoundException

2016-10-26 Thread srowen
Repository: spark Updated Branches: refs/heads/master 6c7d094ec -> 297813647 [SPARK-18027][YARN] .sparkStaging not clean on RM ApplicationNotFoundException ## What changes were proposed in this pull request? Cleanup YARN staging dir on all `KILLED`/`FAILED` paths in `monitorApplication` ## H

spark git commit: [SPARK-18022][SQL] java.lang.NullPointerException instead of real exception when saving DF to MySQL

2016-10-26 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 912487eaf -> c2cce2e60 [SPARK-18022][SQL] java.lang.NullPointerException instead of real exception when saving DF to MySQL ## What changes were proposed in this pull request? On null next exception in JDBC, don't init it as cause or s

spark git commit: [SPARK-18022][SQL] java.lang.NullPointerException instead of real exception when saving DF to MySQL

2016-10-26 Thread srowen
Repository: spark Updated Branches: refs/heads/master 93b8ad184 -> 6c7d094ec [SPARK-18022][SQL] java.lang.NullPointerException instead of real exception when saving DF to MySQL ## What changes were proposed in this pull request? On null next exception in JDBC, don't init it as cause or suppr

spark git commit: [SPARK-17693][SQL] Fixed Insert Failure To Data Source Tables when the Schema has the Comment Field

2016-10-26 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 12b3e8d2e -> 93b8ad184 [SPARK-17693][SQL] Fixed Insert Failure To Data Source Tables when the Schema has the Comment Field ### What changes were proposed in this pull request? ```SQL CREATE TABLE tab1(col1 int COMMENT 'a', col2 int) USING