[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-12-03 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/23037 I thought there was already one for that Hive suite failing... SPARK-23622? --- - To unsubscribe, e-mail:

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-12-03 Thread AzureQ
Github user AzureQ commented on the issue: https://github.com/apache/spark/pull/23037 > ok, I give up on flaky tests. > > Merging to master. Thanks! --- - To unsubscribe, e-mail:

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-12-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/23037 @vanzin Could you create a JIRA for this flaky test? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-12-03 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/23037 ok, I give up on flaky tests. Merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-12-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23037 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99623/ Test FAILed. ---

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-12-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23037 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-12-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23037 **[Test build #99623 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99623/testReport)** for PR 23037 at commit

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-12-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23037 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-12-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23037 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-12-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23037 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/5680/ ---

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-12-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23037 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/5680/ ---

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-12-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23037 **[Test build #99623 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99623/testReport)** for PR 23037 at commit

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-12-03 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/23037 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-12-01 Thread AzureQ
Github user AzureQ commented on the issue: https://github.com/apache/spark/pull/23037 done the change...not sure why the test build errors out with a hive related issue --- - To unsubscribe, e-mail:

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23037 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99530/ Test FAILed. ---

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23037 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23037 **[Test build #99530 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99530/testReport)** for PR 23037 at commit

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23037 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23037 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23037 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/5592/ ---

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23037 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/5592/ ---

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23037 **[Test build #99530 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99530/testReport)** for PR 23037 at commit

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-30 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/23037 Actually I lied. Could you update the `create_dev_build_context` function in `docker-image-tool.sh` to copy this new directory? You can run the script from your build directory to test it.

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-30 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/23037 Merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23037 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/5539/ ---

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23037 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23037 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23037 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99471/ Test PASSed. ---

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23037 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23037 **[Test build #99471 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99471/testReport)** for PR 23037 at commit

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23037 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/5539/ ---

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23037 **[Test build #99471 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99471/testReport)** for PR 23037 at commit

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-29 Thread AzureQ
Github user AzureQ commented on the issue: https://github.com/apache/spark/pull/23037 Cou > When I tried to write automated tests for pyspark in the past it was kind of a pain. It doesn't work the way you expect unless you have a pseudo-terminal, apparently. > >

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-29 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/23037 When I tried to write automated tests for pyspark in the past it was kind of a pain. It doesn't work the way you expect unless you have a pseudo-terminal, apparently. Maybe try to write a

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-29 Thread AzureQ
Github user AzureQ commented on the issue: https://github.com/apache/spark/pull/23037 @vanzin @ifilonenko I'm able to capture stdout of `spark-shell` but not `pyspark` and `sparkR`. Need more investigation on it. Do you happen to know why? ---

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-26 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/23037 Seems like a legitimate failure. ``` name="Run PySpark shell" classname="org.apache.spark.deploy.k8s.integrationtest.KubernetesSuite" time="121.327">

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23037 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23037 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23037 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/5378/ ---

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23037 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99295/ Test PASSed. ---

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23037 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23037 **[Test build #99295 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99295/testReport)** for PR 23037 at commit

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23037 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/5378/ ---

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23037 **[Test build #99295 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99295/testReport)** for PR 23037 at commit

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-26 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/23037 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23037 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23037 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/5181/ ---

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23037 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23037 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23037 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99058/ Test PASSed. ---

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23037 **[Test build #99058 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99058/testReport)** for PR 23037 at commit

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23037 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/5181/ ---

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23037 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23037 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/5154/ ---

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23037 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23037 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23037 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99026/ Test PASSed. ---

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23037 **[Test build #99026 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99026/testReport)** for PR 23037 at commit

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23037 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/5154/ ---

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-19 Thread AzureQ
Github user AzureQ commented on the issue: https://github.com/apache/spark/pull/23037 @ifilonenko I added the test case in `PythonTestsSuites` since it's pyspark specific. --- - To unsubscribe, e-mail:

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23037 **[Test build #99026 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99026/testReport)** for PR 23037 at commit

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23037 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23037 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-15 Thread AzureQ
Github user AzureQ commented on the issue: https://github.com/apache/spark/pull/23037 > > This is fine, but please file a bug. > > Okay, as such, @AzureQ could you add an integration test to `ClientModeTestsSuite` Sure ---

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-15 Thread ifilonenko
Github user ifilonenko commented on the issue: https://github.com/apache/spark/pull/23037 > This is fine, but please file a bug. Okay, as such, @AzureQ could you add an integration test to `ClientModeTestsSuite` ---