spark git commit: [MINOR][DOC] Spacing items in migration guide for readability and consistency

2018-10-18 Thread gurwls223
Repository: spark Updated Branches: refs/heads/branch-2.4 36307b1e4 -> 9ed2e4204 [MINOR][DOC] Spacing items in migration guide for readability and consistency ## What changes were proposed in this pull request? Currently, migration guide has no space between each item which looks too

spark git commit: [MINOR][DOC] Spacing items in migration guide for readability and consistency

2018-10-18 Thread gurwls223
Repository: spark Updated Branches: refs/heads/master 1e6c1d8bf -> c8f7691c6 [MINOR][DOC] Spacing items in migration guide for readability and consistency ## What changes were proposed in this pull request? Currently, migration guide has no space between each item which looks too compact

svn commit: r30153 - in /dev/spark/2.4.1-SNAPSHOT-2018_10_18_22_02-36307b1-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-10-18 Thread pwendell
Author: pwendell Date: Fri Oct 19 05:16:45 2018 New Revision: 30153 Log: Apache Spark 2.4.1-SNAPSHOT-2018_10_18_22_02-36307b1 docs [This commit notification would consist of 1478 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-25493][SQL] Use auto-detection for CRLF in CSV datasource multiline mode

2018-10-18 Thread gurwls223
Repository: spark Updated Branches: refs/heads/master d0ecff285 -> 1e6c1d8bf [SPARK-25493][SQL] Use auto-detection for CRLF in CSV datasource multiline mode ## What changes were proposed in this pull request? CSVs with windows style crlf ('\r\n') don't work in multiline mode. They work fine

spark git commit: [SPARK-25764][ML][EXAMPLES] Update BisectingKMeans example to use ClusteringEvaluator

2018-10-18 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.4 fd5b24726 -> 36307b1e4 [SPARK-25764][ML][EXAMPLES] Update BisectingKMeans example to use ClusteringEvaluator ## What changes were proposed in this pull request? The PR updates the examples for `BisectingKMeans` so that they don't use

spark git commit: [SPARK-25764][ML][EXAMPLES] Update BisectingKMeans example to use ClusteringEvaluator

2018-10-18 Thread wenchen
Repository: spark Updated Branches: refs/heads/master f704ebe90 -> d0ecff285 [SPARK-25764][ML][EXAMPLES] Update BisectingKMeans example to use ClusteringEvaluator ## What changes were proposed in this pull request? The PR updates the examples for `BisectingKMeans` so that they don't use the

svn commit: r30151 - in /dev/spark/2.3.3-SNAPSHOT-2018_10_18_18_02-61b301c-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-10-18 Thread pwendell
Author: pwendell Date: Fri Oct 19 01:16:14 2018 New Revision: 30151 Log: Apache Spark 2.3.3-SNAPSHOT-2018_10_18_18_02-61b301c docs [This commit notification would consist of 1443 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-25683][CORE] Updated the log for the firstTime event Drop occurs

2018-10-18 Thread vanzin
Repository: spark Updated Branches: refs/heads/master 987f38658 -> f704ebe90 [SPARK-25683][CORE] Updated the log for the firstTime event Drop occurs ## What changes were proposed in this pull request? When the first dropEvent occurs, LastReportTimestamp was printing in the log as Wed Dec 31

svn commit: r30144 - in /dev/spark/2.4.1-SNAPSHOT-2018_10_18_14_02-fd5b247-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-10-18 Thread pwendell
Author: pwendell Date: Thu Oct 18 21:17:11 2018 New Revision: 30144 Log: Apache Spark 2.4.1-SNAPSHOT-2018_10_18_14_02-fd5b247 docs [This commit notification would consist of 1478 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-24499][DOC][FOLLOW-UP] Split the page of sql-programming-guide.html to multiple separate pages

2018-10-18 Thread dongjoon
Repository: spark Updated Branches: refs/heads/branch-2.4 715355164 -> fd5b24726 [SPARK-24499][DOC][FOLLOW-UP] Split the page of sql-programming-guide.html to multiple separate pages ## What changes were proposed in this pull request? Forgot to clean remove the link for `Upgrading From Spark

spark git commit: [SPARK-21402][SQL][BACKPORT-2.2] Fix java array of structs deserialization

2018-10-18 Thread dongjoon
Repository: spark Updated Branches: refs/heads/branch-2.2 773c8236c -> 2e3b923e0 [SPARK-21402][SQL][BACKPORT-2.2] Fix java array of structs deserialization This PR is to backport #22708 to branch 2.2. ## What changes were proposed in this pull request? MapObjects expression is used to map

svn commit: r30136 - in /dev/spark/3.0.0-SNAPSHOT-2018_10_18_12_02-987f386-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-10-18 Thread pwendell
Author: pwendell Date: Thu Oct 18 19:17:29 2018 New Revision: 30136 Log: Apache Spark 3.0.0-SNAPSHOT-2018_10_18_12_02-987f386 docs [This commit notification would consist of 1484 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

[2/4] spark git commit: [SPARK-24499][SQL][DOC] Split the page of sql-programming-guide.html to multiple separate pages

2018-10-18 Thread lixiao
http://git-wip-us.apache.org/repos/asf/spark/blob/71535516/docs/sql-programming-guide.md -- diff --git a/docs/sql-programming-guide.md b/docs/sql-programming-guide.md index e45e50d..42b00c9 100644 ---

[1/4] spark git commit: [SPARK-24499][SQL][DOC] Split the page of sql-programming-guide.html to multiple separate pages

2018-10-18 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.4 71a6a9ce8 -> 715355164 http://git-wip-us.apache.org/repos/asf/spark/blob/71535516/docs/sql-pyspark-pandas-with-arrow.md -- diff --git

[3/4] spark git commit: [SPARK-24499][SQL][DOC] Split the page of sql-programming-guide.html to multiple separate pages

2018-10-18 Thread lixiao
http://git-wip-us.apache.org/repos/asf/spark/blob/71535516/docs/sql-data-sources-troubleshooting.md -- diff --git a/docs/sql-data-sources-troubleshooting.md b/docs/sql-data-sources-troubleshooting.md new file mode 100644 index

[4/4] spark git commit: [SPARK-24499][SQL][DOC] Split the page of sql-programming-guide.html to multiple separate pages

2018-10-18 Thread lixiao
[SPARK-24499][SQL][DOC] Split the page of sql-programming-guide.html to multiple separate pages 1. Split the main page of sql-programming-guide into 7 parts: - Getting Started - Data Sources - Performance Turing - Distributed SQL Engine - PySpark Usage Guide for Pandas with Apache Arrow -

[3/4] spark git commit: [SPARK-24499][SQL][DOC] Split the page of sql-programming-guide.html to multiple separate pages

2018-10-18 Thread lixiao
http://git-wip-us.apache.org/repos/asf/spark/blob/987f3865/docs/sql-data-sources-troubleshooting.md -- diff --git a/docs/sql-data-sources-troubleshooting.md b/docs/sql-data-sources-troubleshooting.md new file mode 100644 index

[4/4] spark git commit: [SPARK-24499][SQL][DOC] Split the page of sql-programming-guide.html to multiple separate pages

2018-10-18 Thread lixiao
[SPARK-24499][SQL][DOC] Split the page of sql-programming-guide.html to multiple separate pages ## What changes were proposed in this pull request? 1. Split the main page of sql-programming-guide into 7 parts: - Getting Started - Data Sources - Performance Turing - Distributed SQL Engine -

[1/4] spark git commit: [SPARK-24499][SQL][DOC] Split the page of sql-programming-guide.html to multiple separate pages

2018-10-18 Thread lixiao
Repository: spark Updated Branches: refs/heads/master c2962546d -> 987f38658 http://git-wip-us.apache.org/repos/asf/spark/blob/987f3865/docs/sql-pyspark-pandas-with-arrow.md -- diff --git

[2/4] spark git commit: [SPARK-24499][SQL][DOC] Split the page of sql-programming-guide.html to multiple separate pages

2018-10-18 Thread lixiao
http://git-wip-us.apache.org/repos/asf/spark/blob/987f3865/docs/sql-programming-guide.md -- diff --git a/docs/sql-programming-guide.md b/docs/sql-programming-guide.md index fb03ed2..42b00c9 100644 ---

spark git commit: [SPARK-25758][ML] Deprecate computeCost on BisectingKMeans

2018-10-18 Thread dongjoon
Repository: spark Updated Branches: refs/heads/master 15524c41b -> c2962546d [SPARK-25758][ML] Deprecate computeCost on BisectingKMeans ## What changes were proposed in this pull request? The PR proposes to deprecate the `computeCost` method on `BisectingKMeans` in favor of the adoption of

spark git commit: [SPARK-25758][ML] Deprecate computeCost on BisectingKMeans

2018-10-18 Thread dongjoon
Repository: spark Updated Branches: refs/heads/branch-2.4 ac9a6f08a -> 71a6a9ce8 [SPARK-25758][ML] Deprecate computeCost on BisectingKMeans ## What changes were proposed in this pull request? The PR proposes to deprecate the `computeCost` method on `BisectingKMeans` in favor of the adoption

spark git commit: [SPARK-25682][K8S] Package example jars in same target for dev and distro images.

2018-10-18 Thread vanzin
Repository: spark Updated Branches: refs/heads/master e80f18dbd -> 15524c41b [SPARK-25682][K8S] Package example jars in same target for dev and distro images. This way the image generated from both environments has the same layout, with just a difference in contents that should not affect

spark git commit: [SPARK-25763][SQL][PYSPARK][TEST] Use more `@contextmanager` to ensure clean-up each test.

2018-10-18 Thread gurwls223
Repository: spark Updated Branches: refs/heads/master 1117fc35f -> e80f18dbd [SPARK-25763][SQL][PYSPARK][TEST] Use more `@contextmanager` to ensure clean-up each test. ## What changes were proposed in this pull request? Currently each test in `SQLTest` in PySpark is not cleaned properly. We

spark git commit: [SPARK-25760][SQL] Set AddJarCommand return empty

2018-10-18 Thread dongjoon
Repository: spark Updated Branches: refs/heads/master 734c6af0d -> 1117fc35f [SPARK-25760][SQL] Set AddJarCommand return empty ## What changes were proposed in this pull request? Only `AddJarCommand` return `0`, the user will be confused about what it means. This PR sets it to empty.

svn commit: r30132 - in /dev/spark/3.0.0-SNAPSHOT-2018_10_18_08_02-734c6af-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-10-18 Thread pwendell
Author: pwendell Date: Thu Oct 18 15:16:52 2018 New Revision: 30132 Log: Apache Spark 3.0.0-SNAPSHOT-2018_10_18_08_02-734c6af docs [This commit notification would consist of 1478 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-24601][FOLLOWUP] Update Jackson to 2.9.6 in Kinesis

2018-10-18 Thread srowen
Repository: spark Updated Branches: refs/heads/master c3eaee776 -> 734c6af0d [SPARK-24601][FOLLOWUP] Update Jackson to 2.9.6 in Kinesis ## What changes were proposed in this pull request? Also update Kinesis SDK's Jackson to match Spark's ## How was this patch tested? Existing tests,

svn commit: r30126 - in /dev/spark/3.0.0-SNAPSHOT-2018_10_18_00_02-c3eaee7-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-10-18 Thread pwendell
Author: pwendell Date: Thu Oct 18 07:17:18 2018 New Revision: 30126 Log: Apache Spark 3.0.0-SNAPSHOT-2018_10_18_00_02-c3eaee7 docs [This commit notification would consist of 1478 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]