spark git commit: [SPARK-22648][K8S] Add documentation covering init containers and secrets

2017-12-27 Thread ueshin
Repository: spark Updated Branches: refs/heads/master 171f6ddad -> ded6d27e4 [SPARK-22648][K8S] Add documentation covering init containers and secrets ## What changes were proposed in this pull request? This PR updates the Kubernetes documentation corresponding to the following features/chan

[2/2] spark git commit: [SPARK-22757][KUBERNETES] Enable use of remote dependencies (http, s3, gcs, etc.) in Kubernetes mode

2017-12-27 Thread ueshin
[SPARK-22757][KUBERNETES] Enable use of remote dependencies (http, s3, gcs, etc.) in Kubernetes mode ## What changes were proposed in this pull request? This PR expands the Kubernetes mode to be able to use remote dependencies on http/https endpoints, GCS, S3, etc. It adds steps for configuring

[1/2] spark git commit: [SPARK-22757][KUBERNETES] Enable use of remote dependencies (http, s3, gcs, etc.) in Kubernetes mode

2017-12-27 Thread ueshin
Repository: spark Updated Branches: refs/heads/master 32ec269d0 -> 171f6ddad http://git-wip-us.apache.org/repos/asf/spark/blob/171f6dda/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/steps/initcontainer/InitContainerMountSecretsStep.scala ---

spark git commit: [SPARK-22909][SS] Move Structured Streaming v2 APIs to streaming folder

2017-12-27 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 568398452 -> 32ec269d0 [SPARK-22909][SS] Move Structured Streaming v2 APIs to streaming folder ## What changes were proposed in this pull request? This PR moves Structured Streaming v2 APIs to streaming folder as following: ``` sql/core/sr

spark git commit: [SPARK-18016][SQL][FOLLOW-UP] Code Generation: Constant Pool Limit - reduce entries for mutable state

2017-12-27 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 753793bc8 -> 568398452 [SPARK-18016][SQL][FOLLOW-UP] Code Generation: Constant Pool Limit - reduce entries for mutable state ## What changes were proposed in this pull request? This PR addresses additional review comments in #19811 ## Ho

svn commit: r23928 - in /dev/spark/2.3.0-SNAPSHOT-2017_12_27_20_01-753793b-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2017-12-27 Thread pwendell
Author: pwendell Date: Thu Dec 28 04:14:58 2017 New Revision: 23928 Log: Apache Spark 2.3.0-SNAPSHOT-2017_12_27_20_01-753793b docs [This commit notification would consist of 1420 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

spark git commit: [SPARK-22899][ML][STREAMING] Fix OneVsRestModel transform on streaming data failed.

2017-12-27 Thread jkbradley
Repository: spark Updated Branches: refs/heads/master 774715d5c -> 753793bc8 [SPARK-22899][ML][STREAMING] Fix OneVsRestModel transform on streaming data failed. ## What changes were proposed in this pull request? Fix OneVsRestModel transform on streaming data failed. ## How was this patch t

svn commit: r23918 - in /dev/spark/2.3.0-SNAPSHOT-2017_12_27_08_01-774715d-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2017-12-27 Thread pwendell
Author: pwendell Date: Wed Dec 27 16:14:35 2017 New Revision: 23918 Log: Apache Spark 2.3.0-SNAPSHOT-2017_12_27_08_01-774715d docs [This commit notification would consist of 1420 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

spark git commit: [SPARK-22904][SQL] Add tests for decimal operations and string casts

2017-12-27 Thread lixiao
Repository: spark Updated Branches: refs/heads/master b8bfce51a -> 774715d5c [SPARK-22904][SQL] Add tests for decimal operations and string casts ## What changes were proposed in this pull request? Test coverage for arithmetic operations leading to: 1. Precision loss 2. Overflow Moreover,

svn commit: r23917 - in /dev/spark/2.3.0-SNAPSHOT-2017_12_27_04_01-b8bfce5-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2017-12-27 Thread pwendell
Author: pwendell Date: Wed Dec 27 12:18:58 2017 New Revision: 23917 Log: Apache Spark 2.3.0-SNAPSHOT-2017_12_27_04_01-b8bfce5 docs [This commit notification would consist of 1420 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

spark git commit: [SPARK-22324][SQL][PYTHON][FOLLOW-UP] Update setup.py file.

2017-12-27 Thread gurwls223
Repository: spark Updated Branches: refs/heads/master 6674acd1e -> b8bfce51a [SPARK-22324][SQL][PYTHON][FOLLOW-UP] Update setup.py file. ## What changes were proposed in this pull request? This is a follow-up pr of #19884 updating setup.py file to add pyarrow dependency. ## How was this pat