svn commit: r26480 - in /dev/spark/2.4.0-SNAPSHOT-2018_04_23_20_01-281c1ca-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-04-23 Thread pwendell
Author: pwendell Date: Tue Apr 24 03:15:43 2018 New Revision: 26480 Log: Apache Spark 2.4.0-SNAPSHOT-2018_04_23_20_01-281c1ca docs [This commit notification would consist of 1460 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-23973][SQL] Remove consecutive Sorts

2018-04-23 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 428b90385 -> 281c1ca0d [SPARK-23973][SQL] Remove consecutive Sorts ## What changes were proposed in this pull request? In SPARK-23375 we introduced the ability of removing `Sort` operation during query optimization if the data is already

spark git commit: [SPARK-24029][CORE] Follow up: set SO_REUSEADDR on the server socket.

2018-04-23 Thread gurwls223
Repository: spark Updated Branches: refs/heads/master c8f3ac69d -> 428b90385 [SPARK-24029][CORE] Follow up: set SO_REUSEADDR on the server socket. "childOption" is for the remote connections, not for the server socket that actually listens for incoming connections. Author: Marcelo Vanzin

svn commit: r26478 - in /dev/spark/2.3.1-SNAPSHOT-2018_04_23_14_01-1c3e820-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-04-23 Thread pwendell
Author: pwendell Date: Mon Apr 23 21:14:56 2018 New Revision: 26478 Log: Apache Spark 2.3.1-SNAPSHOT-2018_04_23_14_01-1c3e820 docs [This commit notification would consist of 1443 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: Revert "[SPARK-23799][SQL] FilterEstimation.evaluateInSet produces devision by zero in a case of empty table with analyzed statistics"

2018-04-23 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.3 8eb9a411d -> 1c3e8205d Revert "[SPARK-23799][SQL] FilterEstimation.evaluateInSet produces devision by zero in a case of empty table with analyzed statistics" This reverts commit c2f4ee7baf07501cc1f8a23dd21d14aea53606c7. Project:

spark git commit: [SPARK-23888][CORE] correct the comment of hasAttemptOnHost()

2018-04-23 Thread irashid
Repository: spark Updated Branches: refs/heads/master e82cb6834 -> c8f3ac69d [SPARK-23888][CORE] correct the comment of hasAttemptOnHost() TaskSetManager.hasAttemptOnHost had a misleading comment. The comment said that it only checked for running tasks, but really it checked for any tasks

spark git commit: [SPARK-11237][ML] Add pmml export for k-means in Spark ML

2018-04-23 Thread dbtsai
Repository: spark Updated Branches: refs/heads/master 770add81c -> e82cb6834 [SPARK-11237][ML] Add pmml export for k-means in Spark ML ## What changes were proposed in this pull request? Adding PMML export to Spark ML's KMeans Model. ## How was this patch tested? New unit test for Spark ML

spark git commit: [SPARK-23004][SS] Ensure StateStore.commit is called only once in a streaming aggregation task

2018-04-23 Thread tdas
Repository: spark Updated Branches: refs/heads/branch-2.3 c2f4ee7ba -> 8eb9a411d [SPARK-23004][SS] Ensure StateStore.commit is called only once in a streaming aggregation task ## What changes were proposed in this pull request? A structured streaming query with a streaming aggregation can

spark git commit: [SPARK-23004][SS] Ensure StateStore.commit is called only once in a streaming aggregation task

2018-04-23 Thread tdas
Repository: spark Updated Branches: refs/heads/master 448d248f8 -> 770add81c [SPARK-23004][SS] Ensure StateStore.commit is called only once in a streaming aggregation task ## What changes were proposed in this pull request? A structured streaming query with a streaming aggregation can throw

svn commit: r26477 - in /dev/spark/2.4.0-SNAPSHOT-2018_04_23_12_01-448d248-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-04-23 Thread pwendell
Author: pwendell Date: Mon Apr 23 19:15:45 2018 New Revision: 26477 Log: Apache Spark 2.4.0-SNAPSHOT-2018_04_23_12_01-448d248 docs [This commit notification would consist of 1460 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-21168] KafkaRDD should always set kafka clientId.

2018-04-23 Thread koeninger
Repository: spark Updated Branches: refs/heads/master 293a0f29e -> 448d248f8 [SPARK-21168] KafkaRDD should always set kafka clientId. [https://issues.apache.org/jira/browse/SPARK-21168](https://issues.apache.org/jira/browse/SPARK-21168) There are no a number of other places that a client ID

svn commit: r26470 - in /dev/spark/2.4.0-SNAPSHOT-2018_04_23_08_02-afbdf42-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-04-23 Thread pwendell
Author: pwendell Date: Mon Apr 23 15:21:00 2018 New Revision: 26470 Log: Apache Spark 2.4.0-SNAPSHOT-2018_04_23_08_02-afbdf42 docs [This commit notification would consist of 1460 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-23589][SQL] ExternalMapToCatalyst should support interpreted execution

2018-04-23 Thread hvanhovell
Repository: spark Updated Branches: refs/heads/master d87d30e4f -> afbdf4273 [SPARK-23589][SQL] ExternalMapToCatalyst should support interpreted execution ## What changes were proposed in this pull request? This pr supported interpreted mode for `ExternalMapToCatalyst`. ## How was this patch

spark git commit: [SPARK-23564][SQL] infer additional filters from constraints for join's children

2018-04-23 Thread wenchen
Repository: spark Updated Branches: refs/heads/master f70f46d1e -> d87d30e4f [SPARK-23564][SQL] infer additional filters from constraints for join's children ## What changes were proposed in this pull request? The existing query constraints framework has 2 steps: 1. propagate constraints

spark git commit: [SPARK-23877][SQL][FOLLOWUP] use PhysicalOperation to simplify the handling of Project and Filter over partitioned relation

2018-04-23 Thread wenchen
Repository: spark Updated Branches: refs/heads/master c3a86faa5 -> f70f46d1e [SPARK-23877][SQL][FOLLOWUP] use PhysicalOperation to simplify the handling of Project and Filter over partitioned relation ## What changes were proposed in this pull request? A followup of