spark git commit: [SPARK-24428][K8S] Fix unused code

2018-07-02 Thread foxish
Repository: spark Updated Branches: refs/heads/master 42815548c -> 85fe1297e [SPARK-24428][K8S] Fix unused code ## What changes were proposed in this pull request? Remove code that is misleading and is a leftover from a previous implementation. ## How was this patch tested? Manually.

spark git commit: [SPARK-24547][K8S] Allow for building spark on k8s docker images without cache and don't forget to push spark-py container.

2018-06-20 Thread foxish
Repository: spark Updated Branches: refs/heads/master 3f4bda728 -> 15747cfd3 [SPARK-24547][K8S] Allow for building spark on k8s docker images without cache and don't forget to push spark-py container. ## What changes were proposed in this pull request?

spark git commit: [SPARK-24232][K8S] Add support for secret env vars

2018-05-31 Thread foxish
Repository: spark Updated Branches: refs/heads/master cc976f6cb -> 21e1fc7d4 [SPARK-24232][K8S] Add support for secret env vars ## What changes were proposed in this pull request? * Allows to refer a secret as an env var. * Introduces new config properties in the form:

spark git commit: [SPARK-24137][K8S] Mount local directories as empty dir volumes.

2018-05-10 Thread foxish
Repository: spark Updated Branches: refs/heads/master f4fed0512 -> 6282fc64e [SPARK-24137][K8S] Mount local directories as empty dir volumes. ## What changes were proposed in this pull request? Drastically improves performance and won't cause Spark applications to fail because they write

[1/3] spark git commit: [SPARK-22839][K8S] Refactor to unify driver and executor pod builder APIs

2018-04-13 Thread foxish
Repository: spark Updated Branches: refs/heads/master 0323e6146 -> a83ae0d9b http://git-wip-us.apache.org/repos/asf/spark/blob/a83ae0d9/resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/features/MountSecretsFeatureStepSuite.scala

[2/3] spark git commit: [SPARK-22839][K8S] Refactor to unify driver and executor pod builder APIs

2018-04-13 Thread foxish
http://git-wip-us.apache.org/repos/asf/spark/blob/a83ae0d9/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/steps/DependencyResolutionStep.scala -- diff --git

[3/3] spark git commit: [SPARK-22839][K8S] Refactor to unify driver and executor pod builder APIs

2018-04-13 Thread foxish
[SPARK-22839][K8S] Refactor to unify driver and executor pod builder APIs ## What changes were proposed in this pull request? Breaks down the construction of driver pods and executor pods in a way that uses a common abstraction for both spark-submit creating the driver and

spark git commit: [SPARK-23668][K8S] Add config option for passing through k8s Pod.spec.imagePullSecrets

2018-04-04 Thread foxish
Repository: spark Updated Branches: refs/heads/master a35523653 -> cccaaa14a [SPARK-23668][K8S] Add config option for passing through k8s Pod.spec.imagePullSecrets ## What changes were proposed in this pull request? Pass through the `imagePullSecrets` option to the k8s pod in order to allow

spark git commit: [SPARK-23285][K8S] Add a config property for specifying physical executor cores

2018-04-02 Thread foxish
hao mccheah foxish Author: Yinan Li <y...@google.com> Author: Yinan Li <liyinan...@gmail.com> Closes #20553 from liyinan926/master. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/fe2b7a45 Tree: http://git-wip-us.a

spark git commit: [SPARK-23618][K8S][BUILD] Initialize BUILD_ARGS in docker-image-tool.sh

2018-03-12 Thread foxish
os/asf/spark/commit/d5b41aea Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/d5b41aea Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/d5b41aea Branch: refs/heads/master Commit: d5b41aea62201cd5b1baad2f68f5fc7eb99c62c5 Parents: b304e07 Author: Jooseong Kim <joose...@pinterest.com> Aut