[GitHub] packtpartner opened a new pull request #172: Update documentation.md

packtpartner opened a new pull request #172: Update documentation.md URL: https://github.com/apache/spark-website/pull/172 *Make sure that you generate site HTML with `jekyll build`, and include the changes to the HTML in your pull request also. See README.md for more information. Please

[spark] Diff for: [GitHub] HyukjinKwon closed pull request #23506: [SPARK-26577][SQL] Add input optimizer when reading Hive table by SparkSQL

diff --git a/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala b/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala index 597eef129f63e..688376e634f9d 100644 --- a/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala +++

svn commit: r31981 - in /dev/spark/3.0.0-SNAPSHOT-2019_01_15_20_08-cf133e6-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

Author: pwendell Date: Wed Jan 16 04:20:58 2019 New Revision: 31981 Log: Apache Spark 3.0.0-SNAPSHOT-2019_01_15_20_08-cf133e6 docs [This commit notification would consist of 1777 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

[spark] Diff for: [GitHub] cloud-fan closed pull request #23521: [SPARK-26604][CORE] Clean up channel registration for StreamManager

diff --git a/common/network-common/src/main/java/org/apache/spark/network/server/ChunkFetchRequestHandler.java b/common/network-common/src/main/java/org/apache/spark/network/server/ChunkFetchRequestHandler.java index f08d8b0f984cf..43c3d23b6304d 100644 ---

[spark] branch master updated (2ebb79b -> cf133e6)

This is an automated email from the ASF dual-hosted git repository. wenchen pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git. from 2ebb79b [SPARK-26350][FOLLOWUP] Add actual verification on new UT introduced on SPARK-26350 add cf133e6

[GitHub] srowen commented on issue #171: Remove old releases from download page. Remove unused 'stable' flag and old info

srowen commented on issue #171: Remove old releases from download page. Remove unused 'stable' flag and old info URL: https://github.com/apache/spark-website/pull/171#issuecomment-454599623 I merged this to ensure that the web page matches what is currently mirrored and in the archives.

[GitHub] srowen closed pull request #171: Remove old releases from download page. Remove unused 'stable' flag and old info

srowen closed pull request #171: Remove old releases from download page. Remove unused 'stable' flag and old info URL: https://github.com/apache/spark-website/pull/171 As this is a foreign pull request (from a fork), the diff has been sent to your commit mailing list,

[spark-website] branch asf-site updated: Remove old releases from download page. Remove unused 'stable' flag and old info

This is an automated email from the ASF dual-hosted git repository. srowen pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/spark-website.git The following commit(s) were added to refs/heads/asf-site by this push: new 31485a3 Remove old releases from

[spark-website] Diff for: [GitHub] srowen closed pull request #171: Remove old releases from download page. Remove unused 'stable' flag and old info

diff --git a/downloads.md b/downloads.md index 9cc5414a4..a4ae5f494 100644 --- a/downloads.md +++ b/downloads.md @@ -26,23 +26,6 @@ $(document).ready(function() { 4. Verify this release using the and [project release KEYS](https://www.apache.org/dist/spark/KEYS). -_Note: Starting version

svn commit: r31978 - in /dev/spark/3.0.0-SNAPSHOT-2019_01_15_15_35-2ebb79b-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

Author: pwendell Date: Tue Jan 15 23:47:29 2019 New Revision: 31978 Log: Apache Spark 3.0.0-SNAPSHOT-2019_01_15_15_35-2ebb79b docs [This commit notification would consist of 1777 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

[spark] Diff for: [GitHub] asfgit closed pull request #23544: [SPARK-26350][FOLLOWUP] Add actual verification on new UT introduced on SPARK-26350

diff --git a/external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchSourceSuite.scala b/external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchSourceSuite.scala index 64020882b306e..cb453846134ed 100644 ---

[spark] branch master updated: [SPARK-26350][FOLLOWUP] Add actual verification on new UT introduced on SPARK-26350

This is an automated email from the ASF dual-hosted git repository. zsxwing pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 2ebb79b [SPARK-26350][FOLLOWUP] Add actual

[spark] Diff for: [GitHub] asfgit closed pull request #23208: [SPARK-25530][SQL] data source v2 API refactor (batch write)

diff --git a/sql/core/src/main/java/org/apache/spark/sql/sources/v2/BatchWriteSupportProvider.java b/sql/core/src/main/java/org/apache/spark/sql/sources/v2/BatchWriteSupportProvider.java deleted file mode 100644 index df439e2c02fe3..0 ---

[spark] branch master updated: [SPARK-25530][SQL] data source v2 API refactor (batch write)

This is an automated email from the ASF dual-hosted git repository. lixiao pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 954ef96 [SPARK-25530][SQL] data source v2 API

[spark] Diff for: [GitHub] srowen closed pull request #17726: [SPARK-17928] [Mesos] No driver.memoryOverhead setting for mesos cluster mode

diff --git a/docs/configuration.md b/docs/configuration.md index ff9b802617f08..b961e5b2f32a6 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -169,7 +169,7 @@ of the most common options to set are: The amount of off-heap memory to be allocated per driver in cluster mode, in

[spark] branch master updated: [SPARK-17928][MESOS] No driver.memoryOverhead setting for mesos cluster mode

This is an automated email from the ASF dual-hosted git repository. srowen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 1b75f3b [SPARK-17928][MESOS] No

[spark] Diff for: [GitHub] asfgit closed pull request #23348: [SPARK-25857][core] Add developer documentation regarding delegation tokens.

diff --git a/core/src/main/scala/org/apache/spark/deploy/security/README.md b/core/src/main/scala/org/apache/spark/deploy/security/README.md new file mode 100644 index 0..c3ef60a231f0e --- /dev/null +++ b/core/src/main/scala/org/apache/spark/deploy/security/README.md @@ -0,0 +1,249 @@

[spark] branch master updated: [SPARK-25857][CORE] Add developer documentation regarding delegation tokens.

This is an automated email from the ASF dual-hosted git repository. vanzin pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 8a54492 [SPARK-25857][CORE] Add developer

svn commit: r31976 - in /dev/spark/3.0.0-SNAPSHOT-2019_01_15_10_51-7296999-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

Author: pwendell Date: Tue Jan 15 19:03:56 2019 New Revision: 31976 Log: Apache Spark 3.0.0-SNAPSHOT-2019_01_15_10_51-7296999 docs [This commit notification would consist of 1775 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

[spark] Diff for: [GitHub] srowen closed pull request #23447: [SPARK-26462][CORE] Use ConfigEntry for hardcoded configs for execution categories

diff --git a/core/src/main/scala/org/apache/spark/HeartbeatReceiver.scala b/core/src/main/scala/org/apache/spark/HeartbeatReceiver.scala index ab0ae55ed357d..67f2c279dca5a 100644 --- a/core/src/main/scala/org/apache/spark/HeartbeatReceiver.scala +++

[spark] branch master updated: [SPARK-26462][CORE] Use ConfigEntry for hardcoded configs for execution categories

This is an automated email from the ASF dual-hosted git repository. srowen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 7296999 [SPARK-26462][CORE] Use ConfigEntry for

[spark] Diff for: [GitHub] asfgit closed pull request #23511: [SPARK-26592][SS] Throw exception when kafka delegation token tried to obtain with proxy user

diff --git a/core/src/main/scala/org/apache/spark/deploy/security/KafkaTokenUtil.scala b/core/src/main/scala/org/apache/spark/deploy/security/KafkaTokenUtil.scala index aec0f72feb3c1..f3638533e1b7d 100644 --- a/core/src/main/scala/org/apache/spark/deploy/security/KafkaTokenUtil.scala +++

[spark] branch master updated: [SPARK-26592][SS] Throw exception when kafka delegation token tried to obtain with proxy user

This is an automated email from the ASF dual-hosted git repository. vanzin pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 5ca45e8 [SPARK-26592][SS] Throw exception when

[GitHub] dongjoon-hyun commented on issue #171: Remove old releases from download page. Remove unused 'stable' flag and old info

dongjoon-hyun commented on issue #171: Remove old releases from download page. Remove unused 'stable' flag and old info URL: https://github.com/apache/spark-website/pull/171#issuecomment-454429410 Oh, I didn't see this PR. Could you rebase please?

[GitHub] dongjoon-hyun merged pull request #170: Update 2.3.1 to use archive

dongjoon-hyun merged pull request #170: Update 2.3.1 to use archive URL: https://github.com/apache/spark-website/pull/170 As this is a foreign pull request (from a fork), the diff has been sent to your commit mailing list, commits@spark.apache.org

[spark-website] Diff for: [GitHub] dongjoon-hyun merged pull request #170: Update 2.3.1 to use archive

diff --git a/js/downloads.js b/js/downloads.js index 560896909..1d4c1e77f 100644 --- a/js/downloads.js +++ b/js/downloads.js @@ -34,7 +34,7 @@ var packagesV9 = [hadoop2p7, hadoop2p6, hadoopFree, scala2p12_hadoopFree, source addRelease("2.4.0", new Date("11/02/2018"), packagesV9, true, true);

[spark-website] branch asf-site updated: Make 2.3.1 to use archive (#170)

This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/spark-website.git The following commit(s) were added to refs/heads/asf-site by this push: new 88472e8 Make 2.3.1 to use archive

[GitHub] dongjoon-hyun commented on issue #170: Update 2.3.1 to use archive

dongjoon-hyun commented on issue #170: Update 2.3.1 to use archive URL: https://github.com/apache/spark-website/pull/170#issuecomment-454428645 Merged. This is an automated message from the Apache Git Service. To respond to

[GitHub] dongjoon-hyun commented on issue #170: Update 2.3.1 to use archive

dongjoon-hyun commented on issue #170: Update 2.3.1 to use archive URL: https://github.com/apache/spark-website/pull/170#issuecomment-454428520 Thank you, @HyukjinKwon . This is an automated message from the Apache Git

svn commit: r31973 - in /release/spark: spark-2.1.3/ spark-2.2.2/ spark-2.3.1/

Author: srowen Date: Tue Jan 15 14:44:15 2019 New Revision: 31973 Log: Remove old Spark releases from dist Removed: release/spark/spark-2.1.3/ release/spark/spark-2.2.2/ release/spark/spark-2.3.1/ - To unsubscribe,

svn commit: r31971 - in /dev/spark: 2.4.0-2018_09_15_15_07-1220ab8-bin/ 2.4.0-2018_09_16_00_38-1220ab8-docs/ 2.4.1-SNAPSHOT-2019_01_14_18_37-743dedb-docs/ 3.0.0-SNAPSHOT-2019_01_13_13_55-c01152d-docs/

Author: srowen Date: Tue Jan 15 14:42:56 2019 New Revision: 31971 Log: Remove more old Spark doc snapshots from dev Removed: dev/spark/2.4.0-2018_09_15_15_07-1220ab8-bin/ dev/spark/2.4.0-2018_09_16_00_38-1220ab8-docs/ dev/spark/2.4.1-SNAPSHOT-2019_01_14_18_37-743dedb-docs/

[GitHub] srowen opened a new pull request #171: Remove old releases from download page. Remove unused 'stable' flag and old info

srowen opened a new pull request #171: Remove old releases from download page. Remove unused 'stable' flag and old info URL: https://github.com/apache/spark-website/pull/171 This is an automated message from the Apache Git

[spark] Diff for: [GitHub] dongjoon-hyun closed pull request #23291: [SPARK-26203][SQL][TEST] Benchmark performance of In and InSet expressions

diff --git a/sql/core/benchmarks/InExpressionBenchmark-results.txt b/sql/core/benchmarks/InExpressionBenchmark-results.txt new file mode 100644 index 0..d2adbded66149 --- /dev/null +++ b/sql/core/benchmarks/InExpressionBenchmark-results.txt @@ -0,0 +1,551 @@

[spark] branch master updated: [SPARK-26203][SQL][TEST] Benchmark performance of In and InSet expressions

This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new b45ff02 [SPARK-26203][SQL][TEST] Benchmark

[GitHub] srowen commented on a change in pull request #169: Add Apache Spark 2.2.3 release news and update links

srowen commented on a change in pull request #169: Add Apache Spark 2.2.3 release news and update links URL: https://github.com/apache/spark-website/pull/169#discussion_r247895304 ## File path: js/downloads.js ## @@ -36,7 +36,8 @@ addRelease("2.4.0", new

[spark] Diff for: [GitHub] liupc closed pull request #23537: [SPARK-26614]Fix speculation kill might cause job failure

diff --git a/core/src/main/scala/org/apache/spark/executor/Executor.scala b/core/src/main/scala/org/apache/spark/executor/Executor.scala index a30a501e5d4a1..f92ea7873f9ce 100644 --- a/core/src/main/scala/org/apache/spark/executor/Executor.scala +++

[GitHub] HyukjinKwon commented on issue #170: Update 2.3.1 to use archive

HyukjinKwon commented on issue #170: Update 2.3.1 to use archive URL: https://github.com/apache/spark-website/pull/170#issuecomment-454337315 +1 This is an automated message from the Apache Git Service. To respond to the

svn commit: r31966 - in /dev/spark/3.0.0-SNAPSHOT-2019_01_15_01_24-a77505d-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

Author: pwendell Date: Tue Jan 15 09:36:22 2019 New Revision: 31966 Log: Apache Spark 3.0.0-SNAPSHOT-2019_01_15_01_24-a77505d docs [This commit notification would consist of 1775 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

[GitHub] dongjoon-hyun commented on issue #170: Update 2.3.1 to use archive

dongjoon-hyun commented on issue #170: Update 2.3.1 to use archive URL: https://github.com/apache/spark-website/pull/170#issuecomment-454301569 cc @felixcheung This is an automated message from the Apache Git Service. To

[GitHub] dongjoon-hyun opened a new pull request #170: Update 2.3.1 to use archive

dongjoon-hyun opened a new pull request #170: Update 2.3.1 to use archive URL: https://github.com/apache/spark-website/pull/170 We need to update 2.3.1 download to use archive and remove it from mirror. This is an automated

[GitHub] felixcheung commented on a change in pull request #169: Add Apache Spark 2.2.3 release news and update links

felixcheung commented on a change in pull request #169: Add Apache Spark 2.2.3 release news and update links URL: https://github.com/apache/spark-website/pull/169#discussion_r247791974 ## File path: js/downloads.js ## @@ -36,7 +36,8 @@ addRelease("2.4.0", new

[GitHub] felixcheung commented on a change in pull request #169: Add Apache Spark 2.2.3 release news and update links

felixcheung commented on a change in pull request #169: Add Apache Spark 2.2.3 release news and update links URL: https://github.com/apache/spark-website/pull/169#discussion_r247791974 ## File path: js/downloads.js ## @@ -36,7 +36,8 @@ addRelease("2.4.0", new