spark git commit: [SPARK-23529][K8S] Support mounting volumes

2018-07-10 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master 74a8d6308 -> 5ff1b9ba1 [SPARK-23529][K8S] Support mounting volumes This PR continues #21095 and intersects with #21238. I've added volume mounts as a separate step and added PersistantVolumeClaim support. There is a fundamental problem

[spark] Git Push Summary

2018-07-10 Thread jshao
Repository: spark Updated Tags: refs/tags/v2.3.2-rc2 [created] 307499e1a - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

[2/2] spark git commit: Preparing development version 2.3.3-SNAPSHOT

2018-07-10 Thread jshao
Preparing development version 2.3.3-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/86457a16 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/86457a16 Diff:

[1/2] spark git commit: Preparing Spark release v2.3.2-rc2

2018-07-10 Thread jshao
Repository: spark Updated Branches: refs/heads/branch-2.3 19542f5de -> 86457a16d Preparing Spark release v2.3.2-rc2 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/307499e1 Tree:

svn commit: r28049 - in /dev/spark/2.3.3-SNAPSHOT-2018_07_10_22_01-19542f5-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-07-10 Thread pwendell
Author: pwendell Date: Wed Jul 11 05:16:16 2018 New Revision: 28049 Log: Apache Spark 2.3.3-SNAPSHOT-2018_07_10_22_01-19542f5 docs [This commit notification would consist of 1443 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-24165][SQL] Fixing conditional expressions to handle nullability of nested types

2018-07-10 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 1f94bf492 -> 74a8d6308 [SPARK-24165][SQL] Fixing conditional expressions to handle nullability of nested types ## What changes were proposed in this pull request? This PR is proposing a fix for the output data type of ```If``` and

svn commit: r28048 - in /dev/spark/2.4.0-SNAPSHOT-2018_07_10_20_01-1f94bf4-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-07-10 Thread pwendell
Author: pwendell Date: Wed Jul 11 03:15:48 2018 New Revision: 28048 Log: Apache Spark 2.4.0-SNAPSHOT-2018_07_10_20_01-1f94bf4 docs [This commit notification would consist of 1467 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-24530][PYTHON] Add a control to force Python version in Sphinx via environment variable, SPHINXPYTHON

2018-07-10 Thread gurwls223
Repository: spark Updated Branches: refs/heads/branch-2.3 72eb97ce9 -> 19542f5de [SPARK-24530][PYTHON] Add a control to force Python version in Sphinx via environment variable, SPHINXPYTHON ## What changes were proposed in this pull request? This PR proposes to add `SPHINXPYTHON`

spark git commit: [SPARK-24530][PYTHON] Add a control to force Python version in Sphinx via environment variable, SPHINXPYTHON

2018-07-10 Thread gurwls223
Repository: spark Updated Branches: refs/heads/master 6078b891d -> 1f94bf492 [SPARK-24530][PYTHON] Add a control to force Python version in Sphinx via environment variable, SPHINXPYTHON ## What changes were proposed in this pull request? This PR proposes to add `SPHINXPYTHON` environment

spark git commit: [SPARK-24730][SS] Add policy to choose max as global watermark when streaming query has multiple watermarks

2018-07-10 Thread tdas
Repository: spark Updated Branches: refs/heads/master 32cb50835 -> 6078b891d [SPARK-24730][SS] Add policy to choose max as global watermark when streaming query has multiple watermarks ## What changes were proposed in this pull request? Currently, when a streaming query has multiple

svn commit: r28045 - in /dev/spark/2.4.0-SNAPSHOT-2018_07_10_12_01-32cb508-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-07-10 Thread pwendell
Author: pwendell Date: Tue Jul 10 19:16:13 2018 New Revision: 28045 Log: Apache Spark 2.4.0-SNAPSHOT-2018_07_10_12_01-32cb508 docs [This commit notification would consist of 1467 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-24662][SQL][SS] Support limit in structured streaming

2018-07-10 Thread tdas
Repository: spark Updated Branches: refs/heads/master e0559f238 -> 32cb50835 [SPARK-24662][SQL][SS] Support limit in structured streaming ## What changes were proposed in this pull request? Support the LIMIT operator in structured streaming. For streams in append or complete output mode, a

svn commit: r28035 - in /dev/spark/2.4.0-SNAPSHOT-2018_07_10_08_01-6fe3286-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-07-10 Thread pwendell
Author: pwendell Date: Tue Jul 10 15:16:05 2018 New Revision: 28035 Log: Apache Spark 2.4.0-SNAPSHOT-2018_07_10_08_01-6fe3286 docs [This commit notification would consist of 1467 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-21743][SQL][FOLLOWUP] free aggregate map when task ends

2018-07-10 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 6fe32869c -> e0559f238 [SPARK-21743][SQL][FOLLOWUP] free aggregate map when task ends ## What changes were proposed in this pull request? This is the first follow-up of https://github.com/apache/spark/pull/21573 , which was only merged

spark git commit: [SPARK-24678][SPARK-STREAMING] Give priority in use of 'PROCESS_LOCAL' for spark-streaming

2018-07-10 Thread jshao
Repository: spark Updated Branches: refs/heads/master a28900956 -> 6fe32869c [SPARK-24678][SPARK-STREAMING] Give priority in use of 'PROCESS_LOCAL' for spark-streaming ## What changes were proposed in this pull request? Currently, `BlockRDD.getPreferredLocations` only get hosts info of

svn commit: r28023 - in /dev/spark/2.4.0-SNAPSHOT-2018_07_10_04_01-a289009-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-07-10 Thread pwendell
Author: pwendell Date: Tue Jul 10 11:17:52 2018 New Revision: 28023 Log: Apache Spark 2.4.0-SNAPSHOT-2018_07_10_04_01-a289009 docs [This commit notification would consist of 1467 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-24706][SQL] ByteType and ShortType support pushdown to parquet

2018-07-10 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 4984f1af7 -> a28900956 [SPARK-24706][SQL] ByteType and ShortType support pushdown to parquet ## What changes were proposed in this pull request? `ByteType` and `ShortType` support pushdown to parquet data source. [Benchmark

svn commit: r28019 - in /dev/spark/2.4.0-SNAPSHOT-2018_07_10_00_01-4984f1a-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-07-10 Thread pwendell
Author: pwendell Date: Tue Jul 10 07:16:39 2018 New Revision: 28019 Log: Apache Spark 2.4.0-SNAPSHOT-2018_07_10_00_01-4984f1a docs [This commit notification would consist of 1467 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]