[GitHub] spark issue #20192: [SPARK-22994][k8s] Use a single image for all Spark cont...

2018-01-11 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20192 Merging to master / 2.3. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: revie

[GitHub] spark issue #20192: [SPARK-22994][k8s] Use a single image for all Spark cont...

2018-01-10 Thread mccheah
Github user mccheah commented on the issue: https://github.com/apache/spark/pull/20192 Good to merge here? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #20192: [SPARK-22994][k8s] Use a single image for all Spark cont...

2018-01-10 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20192 If there's no more feedback I'll merge this later today. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For ad

[GitHub] spark issue #20192: [SPARK-22994][k8s] Use a single image for all Spark cont...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20192 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85870/ Test PASSed. ---

[GitHub] spark issue #20192: [SPARK-22994][k8s] Use a single image for all Spark cont...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20192 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20192: [SPARK-22994][k8s] Use a single image for all Spark cont...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20192 **[Test build #85870 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85870/testReport)** for PR 20192 at commit [`e771ed9`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #20192: [SPARK-22994][k8s] Use a single image for all Spark cont...

2018-01-09 Thread foxish
Github user foxish commented on the issue: https://github.com/apache/spark/pull/20192 Great, thanks @vanzin. We'll probably need to add a test case using the new option as well - I can take care of that. Thanks for the change. --- --

[GitHub] spark issue #20192: [SPARK-22994][k8s] Use a single image for all Spark cont...

2018-01-09 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20192 ``` $ mvn clean integration-test -Dspark-distro-tgz=/work/apache/spark/spark-2.3.0-SNAPSHOT-bin-2.7.3.tgz -DextraScalaTestArgs="-Dspark.kubernetes.test.master=k8s://https://192.168.99.100:8443 -

[GitHub] spark issue #20192: [SPARK-22994][k8s] Use a single image for all Spark cont...

2018-01-09 Thread foxish
Github user foxish commented on the issue: https://github.com/apache/spark/pull/20192 Thanks @vanzin. I was waiting on spark-dev [thread on integration testing](http://apache-spark-developers-list.1001551.n3.nabble.com/Integration-testing-and-Scheduler-Backends-td23105.html) to conclu

[GitHub] spark issue #20192: [SPARK-22994][k8s] Use a single image for all Spark cont...

2018-01-09 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20192 > do you have some time to modify the integration tests as well I can try to look, but really you guys should be putting that code into the Spark repo. I don't see a task under SPARK-18278 fo

[GitHub] spark issue #20192: [SPARK-22994][k8s] Use a single image for all Spark cont...

2018-01-09 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/20192 LGTM. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.a

[GitHub] spark issue #20192: [SPARK-22994][k8s] Use a single image for all Spark cont...

2018-01-09 Thread foxish
Github user foxish commented on the issue: https://github.com/apache/spark/pull/20192 @vanzin, do you have some time to modify the integration tests as well? The change LGTM, but a clean run on minikube would give us a lot more confidence. Until the integration tests get checked in to

[GitHub] spark issue #20192: [SPARK-22994][k8s] Use a single image for all Spark cont...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20192 **[Test build #85870 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85870/testReport)** for PR 20192 at commit [`e771ed9`](https://github.com/apache/spark/commit/e7

[GitHub] spark issue #20192: [SPARK-22994][k8s] Use a single image for all Spark cont...

2018-01-09 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20192 > users with custom docker images can override the classpath by I wrote this in a comment above, but there needs to be a proper definition of how to customize these docker images. There needs

[GitHub] spark issue #20192: [SPARK-22994][k8s] Use a single image for all Spark cont...

2018-01-08 Thread foxish
Github user foxish commented on the issue: https://github.com/apache/spark/pull/20192 Our [integration tests](https://github.com/apache-spark-on-k8s/spark-integration) should be changed to accommodate this modification and test it, and we should also add some new tests utilizing the

[GitHub] spark issue #20192: [SPARK-22994][k8s] Use a single image for all Spark cont...

2018-01-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20192 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20192: [SPARK-22994][k8s] Use a single image for all Spark cont...

2018-01-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20192 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85814/ Test PASSed. ---

[GitHub] spark issue #20192: [SPARK-22994][k8s] Use a single image for all Spark cont...

2018-01-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20192 **[Test build #85814 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85814/testReport)** for PR 20192 at commit [`bfad831`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #20192: [SPARK-22994][k8s] Use a single image for all Spark cont...

2018-01-08 Thread mccheah
Github user mccheah commented on the issue: https://github.com/apache/spark/pull/20192 +1 - users with custom docker images can override the classpath by putting different contents in the `jars` directory or by using a custom `SPARK_CLASSPATH` environment variable in the child image.

[GitHub] spark issue #20192: [SPARK-22994][k8s] Use a single image for all Spark cont...

2018-01-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20192 **[Test build #85814 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85814/testReport)** for PR 20192 at commit [`bfad831`](https://github.com/apache/spark/commit/bf