svn commit: r30733 - in /dev/spark/3.0.0-SNAPSHOT-2018_11_06_00_02-3ed91c9-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-11-06 Thread pwendell
Author: pwendell Date: Tue Nov 6 08:17:10 2018 New Revision: 30733 Log: Apache Spark 3.0.0-SNAPSHOT-2018_11_06_00_02-3ed91c9 docs [This commit notification would consist of 1471 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

spark git commit: [SPARK-22148][SPARK-15815][SCHEDULER] Acquire new executors to avoid hang because of blacklisting

2018-11-06 Thread tgraves
Repository: spark Updated Branches: refs/heads/master 3ed91c9b8 -> fdd3bace1 [SPARK-22148][SPARK-15815][SCHEDULER] Acquire new executors to avoid hang because of blacklisting ## What changes were proposed in this pull request? Every time a task is unschedulable because of the condition where

spark git commit: [SPARK-22148][SPARK-15815][SCHEDULER] Acquire new executors to avoid hang because of blacklisting

2018-11-06 Thread tgraves
Repository: spark Updated Branches: refs/heads/branch-2.4 f98c0ad02 -> 52e9711d0 [SPARK-22148][SPARK-15815][SCHEDULER] Acquire new executors to avoid hang because of blacklisting ## What changes were proposed in this pull request? Every time a task is unschedulable because of the condition wh

spark git commit: [SPARK-25866][ML] Update KMeans formatVersion

2018-11-06 Thread wenchen
Repository: spark Updated Branches: refs/heads/master fdd3bace1 -> 6b425874d [SPARK-25866][ML] Update KMeans formatVersion ## What changes were proposed in this pull request? When we added the `distanceMeasure`, we didn't update the `formatVersion` for `KMeans`. Despite this is not a big iss

spark git commit: [SPARK-25866][ML] Update KMeans formatVersion

2018-11-06 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.4 52e9711d0 -> d6c713fa5 [SPARK-25866][ML] Update KMeans formatVersion ## What changes were proposed in this pull request? When we added the `distanceMeasure`, we didn't update the `formatVersion` for `KMeans`. Despite this is not a big

svn commit: r30738 - in /dev/spark/3.0.0-SNAPSHOT-2018_11_06_08_04-6b42587-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-11-06 Thread pwendell
Author: pwendell Date: Tue Nov 6 16:18:36 2018 New Revision: 30738 Log: Apache Spark 3.0.0-SNAPSHOT-2018_11_06_08_04-6b42587 docs [This commit notification would consist of 1471 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

svn commit: r30740 - in /dev/spark/2.4.1-SNAPSHOT-2018_11_06_10_05-d6c713f-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-11-06 Thread pwendell
Author: pwendell Date: Tue Nov 6 18:19:31 2018 New Revision: 30740 Log: Apache Spark 2.4.1-SNAPSHOT-2018_11_06_10_05-d6c713f docs [This commit notification would consist of 1476 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

spark git commit: [SPARK-25871][STREAMING] Don't use EC for streaming WAL

2018-11-06 Thread vanzin
Repository: spark Updated Branches: refs/heads/master 6b425874d -> cee230160 [SPARK-25871][STREAMING] Don't use EC for streaming WAL The write ahead log expects to be able to call hflush, but that is a no-op when writing to a file with hdfs erasure coding. So ensure that file is always writ

svn commit: r30742 - in /dev/spark/3.0.0-SNAPSHOT-2018_11_06_12_03-cee2301-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-11-06 Thread pwendell
Author: pwendell Date: Tue Nov 6 20:17:26 2018 New Revision: 30742 Log: Apache Spark 3.0.0-SNAPSHOT-2018_11_06_12_03-cee2301 docs [This commit notification would consist of 1471 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

spark git commit: [MINOR] update known_translations

2018-11-06 Thread lixiao
Repository: spark Updated Branches: refs/heads/master cee230160 -> a241a150d [MINOR] update known_translations ## What changes were proposed in this pull request? update known_translations after running `translate-contributors.py` during 2.4.0 release ## How was this patch tested? N/A Clo

spark git commit: [MINOR] update known_translations

2018-11-06 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.4 d6c713fa5 -> 4c91b224a [MINOR] update known_translations ## What changes were proposed in this pull request? update known_translations after running `translate-contributors.py` during 2.4.0 release ## How was this patch tested? N/A

spark git commit: [SPARK-25676][SQL][TEST] Rename and refactor BenchmarkWideTable to use main method

2018-11-06 Thread dongjoon
Repository: spark Updated Branches: refs/heads/master a241a150d -> 63ca4bbe7 [SPARK-25676][SQL][TEST] Rename and refactor BenchmarkWideTable to use main method ## What changes were proposed in this pull request? Refactor BenchmarkWideTable to use main method. Generate benchmark result: ```

svn commit: r30743 - in /dev/spark/3.0.0-SNAPSHOT-2018_11_06_16_02-63ca4bb-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-11-06 Thread pwendell
Author: pwendell Date: Wed Nov 7 00:16:51 2018 New Revision: 30743 Log: Apache Spark 3.0.0-SNAPSHOT-2018_11_06_16_02-63ca4bb docs [This commit notification would consist of 1471 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

svn commit: r30744 - in /dev/spark/2.4.1-SNAPSHOT-2018_11_06_18_02-4c91b22-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-11-06 Thread pwendell
Author: pwendell Date: Wed Nov 7 02:17:14 2018 New Revision: 30744 Log: Apache Spark 2.4.1-SNAPSHOT-2018_11_06_18_02-4c91b22 docs [This commit notification would consist of 1476 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

spark-website git commit: update the release-process document

2018-11-06 Thread wenchen
Repository: spark-website Updated Branches: refs/heads/asf-site 005a2a0d1 -> 44d255281 update the release-process document Our document of doing release is out-dated. This PR updates the release doc to match the process of Spark 2.4.0 release. 1. cutting RC has been fully automated. People sh

spark git commit: [SPARK-25950][SQL] from_csv should respect to spark.sql.columnNameOfCorruptRecord

2018-11-06 Thread gurwls223
Repository: spark Updated Branches: refs/heads/master 63ca4bbe7 -> 76813cfa1 [SPARK-25950][SQL] from_csv should respect to spark.sql.columnNameOfCorruptRecord ## What changes were proposed in this pull request? Fix for `CsvToStructs` to take into account SQL config `spark.sql.columnNameOfCo

svn commit: r30745 - in /dev/spark/3.0.0-SNAPSHOT-2018_11_06_20_02-76813cf-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-11-06 Thread pwendell
Author: pwendell Date: Wed Nov 7 04:17:12 2018 New Revision: 30745 Log: Apache Spark 3.0.0-SNAPSHOT-2018_11_06_20_02-76813cf docs [This commit notification would consist of 1471 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

spark git commit: [SPARK-25098][SQL] Trim the string when cast stringToTimestamp and stringToDate

2018-11-06 Thread dongjoon
Repository: spark Updated Branches: refs/heads/master 76813cfa1 -> 9e9fa2f69 [SPARK-25098][SQL] Trim the string when cast stringToTimestamp and stringToDate ## What changes were proposed in this pull request? **Hive** and **Oracle** trim the string when cast `stringToTimestamp` and `stringTo