svn commit: r24148 - in /dev/spark/v2.3.0-rc1-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _site/api/java/org/apache/spark

2018-01-11 Thread sameerag
Author: sameerag Date: Fri Jan 12 07:52:52 2018 New Revision: 24148 Log: Apache Spark v2.3.0-rc1 docs [This commit notification would consist of 1430 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-22986][CORE] Use a cache to avoid instantiating multiple instances of broadcast variable values

2018-01-11 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.3 55695c712 -> 3ae3e1bb7 [SPARK-22986][CORE] Use a cache to avoid instantiating multiple instances of broadcast variable values When resources happen to be constrained on an executor the first time a broadcast variable is instantiated

spark git commit: [SPARK-22986][CORE] Use a cache to avoid instantiating multiple instances of broadcast variable values

2018-01-11 Thread wenchen
Repository: spark Updated Branches: refs/heads/master b5042d75c -> cbe7c6fbf [SPARK-22986][CORE] Use a cache to avoid instantiating multiple instances of broadcast variable values When resources happen to be constrained on an executor the first time a broadcast variable is instantiated it

svn commit: r24147 - /dev/spark/v2.3.0-rc1-bin/

2018-01-11 Thread sameerag
Author: sameerag Date: Fri Jan 12 07:25:00 2018 New Revision: 24147 Log: Apache Spark v2.3.0-rc1 Added: dev/spark/v2.3.0-rc1-bin/ dev/spark/v2.3.0-rc1-bin/SparkR_2.3.0.tar.gz (with props) dev/spark/v2.3.0-rc1-bin/SparkR_2.3.0.tar.gz.asc

svn commit: r24145 - in /dev/spark/2.3.0-SNAPSHOT-2018_01_11_20_01-b5042d7-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-01-11 Thread pwendell
Author: pwendell Date: Fri Jan 12 04:15:08 2018 New Revision: 24145 Log: Apache Spark 2.3.0-SNAPSHOT-2018_01_11_20_01-b5042d7 docs [This commit notification would consist of 1440 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

svn commit: r24143 - in /dev/spark/2.3.1-SNAPSHOT-2018_01_11_18_01-55695c7-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-01-11 Thread pwendell
Author: pwendell Date: Fri Jan 12 02:15:01 2018 New Revision: 24143 Log: Apache Spark 2.3.1-SNAPSHOT-2018_01_11_18_01-55695c7 docs [This commit notification would consist of 1440 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-23008][ML] OnehotEncoderEstimator python API

2018-01-11 Thread jkbradley
Repository: spark Updated Branches: refs/heads/branch-2.3 6bb22961c -> 55695c712 [SPARK-23008][ML] OnehotEncoderEstimator python API ## What changes were proposed in this pull request? OnehotEncoderEstimator python API. ## How was this patch tested? doctest Author: WeichenXu

spark git commit: [SPARK-23008][ML] OnehotEncoderEstimator python API

2018-01-11 Thread jkbradley
Repository: spark Updated Branches: refs/heads/master 186bf8fb2 -> b5042d75c [SPARK-23008][ML] OnehotEncoderEstimator python API ## What changes were proposed in this pull request? OnehotEncoderEstimator python API. ## How was this patch tested? doctest Author: WeichenXu

svn commit: r24140 - in /dev/spark/2.3.0-SNAPSHOT-2018_01_11_16_01-186bf8f-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-01-11 Thread pwendell
Author: pwendell Date: Fri Jan 12 00:14:59 2018 New Revision: 24140 Log: Apache Spark 2.3.0-SNAPSHOT-2018_01_11_16_01-186bf8f docs [This commit notification would consist of 1440 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

[2/2] spark git commit: Preparing development version 2.3.1-SNAPSHOT

2018-01-11 Thread sameerag
Preparing development version 2.3.1-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/6bb22961 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/6bb22961 Diff:

[1/2] spark git commit: Preparing Spark release v2.3.0-rc1

2018-01-11 Thread sameerag
Repository: spark Updated Branches: refs/heads/branch-2.3 2ec302658 -> 6bb22961c Preparing Spark release v2.3.0-rc1 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/964cc2e3 Tree:

[spark] Git Push Summary

2018-01-11 Thread sameerag
Repository: spark Updated Tags: refs/tags/v2.3.0-rc1 [created] 964cc2e31 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

svn commit: r24138 - in /dev/spark/2.3.0-SNAPSHOT-2018_01_11_14_01-2ec3026-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-01-11 Thread pwendell
Author: pwendell Date: Thu Jan 11 22:15:09 2018 New Revision: 24138 Log: Apache Spark 2.3.0-SNAPSHOT-2018_01_11_14_01-2ec3026 docs [This commit notification would consist of 1440 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-23046][ML][SPARKR] Have RFormula include VectorSizeHint in pipeline

2018-01-11 Thread jkbradley
Repository: spark Updated Branches: refs/heads/branch-2.3 f891ee324 -> 2ec302658 [SPARK-23046][ML][SPARKR] Have RFormula include VectorSizeHint in pipeline ## What changes were proposed in this pull request? Including VectorSizeHint in RFormula piplelines will allow them to be applied to

spark git commit: [SPARK-23046][ML][SPARKR] Have RFormula include VectorSizeHint in pipeline

2018-01-11 Thread jkbradley
Repository: spark Updated Branches: refs/heads/master 6f7aaed80 -> 186bf8fb2 [SPARK-23046][ML][SPARKR] Have RFormula include VectorSizeHint in pipeline ## What changes were proposed in this pull request? Including VectorSizeHint in RFormula piplelines will allow them to be applied to

svn commit: r24135 - in /dev/spark/2.3.0-SNAPSHOT-2018_01_11_12_01-6f7aaed-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-01-11 Thread pwendell
Author: pwendell Date: Thu Jan 11 20:15:08 2018 New Revision: 24135 Log: Apache Spark 2.3.0-SNAPSHOT-2018_01_11_12_01-6f7aaed docs [This commit notification would consist of 1440 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

[1/2] spark git commit: [SPARK-22908] Add kafka source and sink for continuous processing.

2018-01-11 Thread tdas
Repository: spark Updated Branches: refs/heads/branch-2.3 b94debd2b -> f891ee324 http://git-wip-us.apache.org/repos/asf/spark/blob/f891ee32/sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala -- diff --git

[2/2] spark git commit: [SPARK-22908] Add kafka source and sink for continuous processing.

2018-01-11 Thread tdas
[SPARK-22908] Add kafka source and sink for continuous processing. ## What changes were proposed in this pull request? Add kafka source and sink for continuous processing. This involves two small changes to the execution engine: * Bring data reader close() into the normal data reader thread to

[2/2] spark git commit: [SPARK-22908] Add kafka source and sink for continuous processing.

2018-01-11 Thread tdas
[SPARK-22908] Add kafka source and sink for continuous processing. ## What changes were proposed in this pull request? Add kafka source and sink for continuous processing. This involves two small changes to the execution engine: * Bring data reader close() into the normal data reader thread to

[1/2] spark git commit: [SPARK-22908] Add kafka source and sink for continuous processing.

2018-01-11 Thread tdas
Repository: spark Updated Branches: refs/heads/master 0b2eefb67 -> 6f7aaed80 http://git-wip-us.apache.org/repos/asf/spark/blob/6f7aaed8/sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala -- diff --git

spark git commit: [SPARK-22994][K8S] Use a single image for all Spark containers.

2018-01-11 Thread vanzin
Repository: spark Updated Branches: refs/heads/branch-2.3 f624850fe -> b94debd2b [SPARK-22994][K8S] Use a single image for all Spark containers. This change allows a user to submit a Spark application on kubernetes having to provide a single image, instead of one image for each type of

spark git commit: [SPARK-22994][K8S] Use a single image for all Spark containers.

2018-01-11 Thread vanzin
Repository: spark Updated Branches: refs/heads/master 6d230dccf -> 0b2eefb67 [SPARK-22994][K8S] Use a single image for all Spark containers. This change allows a user to submit a Spark application on kubernetes having to provide a single image, instead of one image for each type of container.

svn commit: r24133 - in /dev/spark/2.3.0-SNAPSHOT-2018_01_11_08_01-6d230dc-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-01-11 Thread pwendell
Author: pwendell Date: Thu Jan 11 16:20:31 2018 New Revision: 24133 Log: Apache Spark 2.3.0-SNAPSHOT-2018_01_11_08_01-6d230dc docs [This commit notification would consist of 1440 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: Update PageRank.scala

2018-01-11 Thread srowen
Repository: spark Updated Branches: refs/heads/master b46e58b74 -> 6d230dccf Update PageRank.scala ## What changes were proposed in this pull request? Hi, acording to code below, "if (id == src) (0.0, Double.NegativeInfinity) else (0.0, 0.0)" I think the comment can be wrong ## How was this

svn commit: r24132 - in /dev/spark/2.3.0-SNAPSHOT-2018_01_11_06_01-f624850-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-01-11 Thread pwendell
Author: pwendell Date: Thu Jan 11 14:15:28 2018 New Revision: 24132 Log: Apache Spark 2.3.0-SNAPSHOT-2018_01_11_06_01-f624850 docs [This commit notification would consist of 1440 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-19732][FOLLOW-UP] Document behavior changes made in na.fill and fillna

2018-01-11 Thread gurwls223
Repository: spark Updated Branches: refs/heads/branch-2.3 9ca0f6eaf -> f624850fe [SPARK-19732][FOLLOW-UP] Document behavior changes made in na.fill and fillna ## What changes were proposed in this pull request? https://github.com/apache/spark/pull/18164 introduces the behavior changes. We

spark git commit: [SPARK-19732][FOLLOW-UP] Document behavior changes made in na.fill and fillna

2018-01-11 Thread gurwls223
Repository: spark Updated Branches: refs/heads/master 76892bcf2 -> b46e58b74 [SPARK-19732][FOLLOW-UP] Document behavior changes made in na.fill and fillna ## What changes were proposed in this pull request? https://github.com/apache/spark/pull/18164 introduces the behavior changes. We need

spark git commit: [SPARK-23000][TEST-HADOOP2.6] Fix Flaky test suite DataSourceWithHiveMetastoreCatalogSuite

2018-01-11 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.3 799598905 -> 9ca0f6eaf [SPARK-23000][TEST-HADOOP2.6] Fix Flaky test suite DataSourceWithHiveMetastoreCatalogSuite ## What changes were proposed in this pull request? The Spark 2.3 branch still failed due to the flaky test suite

spark git commit: [SPARK-23000][TEST-HADOOP2.6] Fix Flaky test suite DataSourceWithHiveMetastoreCatalogSuite

2018-01-11 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 0552c36e0 -> 76892bcf2 [SPARK-23000][TEST-HADOOP2.6] Fix Flaky test suite DataSourceWithHiveMetastoreCatalogSuite ## What changes were proposed in this pull request? The Spark 2.3 branch still failed due to the flaky test suite

spark git commit: [SPARK-22967][TESTS] Fix VersionSuite's unit tests by change Windows path into URI path

2018-01-11 Thread gurwls223
Repository: spark Updated Branches: refs/heads/master 1c70da3bf -> 0552c36e0 [SPARK-22967][TESTS] Fix VersionSuite's unit tests by change Windows path into URI path ## What changes were proposed in this pull request? Two unit test will fail due to Windows format path: 1.test(s"$version:

spark git commit: [SPARK-22967][TESTS] Fix VersionSuite's unit tests by change Windows path into URI path

2018-01-11 Thread gurwls223
Repository: spark Updated Branches: refs/heads/branch-2.3 b78130123 -> 799598905 [SPARK-22967][TESTS] Fix VersionSuite's unit tests by change Windows path into URI path ## What changes were proposed in this pull request? Two unit test will fail due to Windows format path:

svn commit: r24131 - in /dev/spark/2.3.0-SNAPSHOT-2018_01_11_04_01-1c70da3-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-01-11 Thread pwendell
Author: pwendell Date: Thu Jan 11 12:17:45 2018 New Revision: 24131 Log: Apache Spark 2.3.0-SNAPSHOT-2018_01_11_04_01-1c70da3 docs [This commit notification would consist of 1440 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

[1/2] spark git commit: [SPARK-20657][CORE] Speed up rendering of the stages page.

2018-01-11 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.3 d9a973d65 -> b78130123 http://git-wip-us.apache.org/repos/asf/spark/blob/b7813012/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala -- diff --git

[2/2] spark git commit: [SPARK-20657][CORE] Speed up rendering of the stages page.

2018-01-11 Thread wenchen
[SPARK-20657][CORE] Speed up rendering of the stages page. There are two main changes to speed up rendering of the tasks list when rendering the stage page. The first one makes the code only load the tasks being shown in the current page of the tasks table, and information related to only those

[2/2] spark git commit: [SPARK-20657][CORE] Speed up rendering of the stages page.

2018-01-11 Thread wenchen
[SPARK-20657][CORE] Speed up rendering of the stages page. There are two main changes to speed up rendering of the tasks list when rendering the stage page. The first one makes the code only load the tasks being shown in the current page of the tasks table, and information related to only those

[1/2] spark git commit: [SPARK-20657][CORE] Speed up rendering of the stages page.

2018-01-11 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 87c98de8b -> 1c70da3bf http://git-wip-us.apache.org/repos/asf/spark/blob/1c70da3b/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala -- diff --git

spark git commit: [SPARK-23001][SQL] Fix NullPointerException when DESC a database with NULL description

2018-01-11 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.2 0d943d96b -> acab4e7a0 [SPARK-23001][SQL] Fix NullPointerException when DESC a database with NULL description ## What changes were proposed in this pull request? When users' DB description is NULL, users might hit

spark git commit: [SPARK-23001][SQL] Fix NullPointerException when DESC a database with NULL description

2018-01-11 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.3 317b0aaed -> d9a973d65 [SPARK-23001][SQL] Fix NullPointerException when DESC a database with NULL description ## What changes were proposed in this pull request? When users' DB description is NULL, users might hit

spark git commit: [SPARK-23001][SQL] Fix NullPointerException when DESC a database with NULL description

2018-01-11 Thread wenchen
Repository: spark Updated Branches: refs/heads/master a6647ffbf -> 87c98de8b [SPARK-23001][SQL] Fix NullPointerException when DESC a database with NULL description ## What changes were proposed in this pull request? When users' DB description is NULL, users might hit `NullPointerException`.