[GitHub] [spark] dongjoon-hyun edited a comment on pull request #28594: [SPARK-31780][K8S][TESTS] Add R test tag to exclude R K8s image building and test
dongjoon-hyun edited a comment on pull request #28594: URL: https://github.com/apache/spark/pull/28594#issuecomment-663593043 Thanks. I understand, @ifilonenko . :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on pull request #28594: [SPARK-31780][K8S][TESTS] Add R test tag to exclude R K8s image building and test
dongjoon-hyun edited a comment on pull request #28594: URL: https://github.com/apache/spark/pull/28594#issuecomment-663288073 I found another evidence from our Apache Spark AmpLab Jenkins farm log. The failed test case is `R test`. - https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/lastUnsuccessfulBuild/console  ``` Started 6 days 0 hr ago Took 1 hr 18 min on research-jenkins-worker-09 ``` ``` PR #28708: [SPARK-20629][CORE][K8S] Co... ``` ``` KubernetesSuite: - Run SparkPi with no resources - Run SparkPi with a very long application name. - Use SparkLauncher.NO_RESOURCE - Run SparkPi with a master URL without a scheme. - Run SparkPi with an argument. - Run SparkPi with custom labels, annotations, and environment variables. - All pods have the same service account by default - Run extraJVMOptions check on driver - Run SparkRemoteFileTest using a remote data file - Run SparkPi with env and mount secrets. - Run PySpark on simple pi.py example - Run PySpark with Python3 to test a pyfiles example - Run PySpark with memory customization - Run in client mode. - Start pod creation from template - PVs with local storage - Launcher client dependencies - Test basic decommissioning - Run SparkR on simple dataframe.R example *** FAILED *** The code passed to eventually never returned normally. Attempted 190 times over 3.000988466737 minutes. Last failure message: false was not true. (KubernetesSuite.scala:386) Run completed in 14 minutes, 46 seconds. Total number of tests run: 19 Suites: completed 2, aborted 0 Tests: succeeded 18, failed 1, canceled 0, ignored 0, pending 0 *** 1 TEST FAILED *** ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on pull request #28594: [SPARK-31780][K8S][TESTS] Add R test tag to exclude R K8s image building and test
dongjoon-hyun edited a comment on pull request #28594: URL: https://github.com/apache/spark/pull/28594#issuecomment-663289180 Since this PR was merged on May 20 and I provided multiple recent evidences of the R testing like the above, @ifilonenko 's claim is wrong. > This has been a side-effect of this change. I'll stop my investigation here. Please take a look at more recent commits to solve your problems. Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on pull request #28594: [SPARK-31780][K8S][TESTS] Add R test tag to exclude R K8s image building and test
dongjoon-hyun edited a comment on pull request #28594: URL: https://github.com/apache/spark/pull/28594#issuecomment-663289180 Since this PR was merged on May 20 and I provided multiple evidences of the R testing like the above, @ifilonenko 's claim is wrong. > This has been a side-effect of this change. I'll stop my investigation here. Please take a look at more recent commits to solve your problems. Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on pull request #28594: [SPARK-31780][K8S][TESTS] Add R test tag to exclude R K8s image building and test
dongjoon-hyun edited a comment on pull request #28594: URL: https://github.com/apache/spark/pull/28594#issuecomment-663289180 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on pull request #28594: [SPARK-31780][K8S][TESTS] Add R test tag to exclude R K8s image building and test
dongjoon-hyun edited a comment on pull request #28594: URL: https://github.com/apache/spark/pull/28594#issuecomment-663288073 I found another evidence from AmpLab Jenkins farm log. The failed test case is `R test`. - https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/lastUnsuccessfulBuild/console  ``` Started 6 days 0 hr ago Took 1 hr 18 min on research-jenkins-worker-09 ``` ``` PR #28708: [SPARK-20629][CORE][K8S] Co... ``` ``` KubernetesSuite: - Run SparkPi with no resources - Run SparkPi with a very long application name. - Use SparkLauncher.NO_RESOURCE - Run SparkPi with a master URL without a scheme. - Run SparkPi with an argument. - Run SparkPi with custom labels, annotations, and environment variables. - All pods have the same service account by default - Run extraJVMOptions check on driver - Run SparkRemoteFileTest using a remote data file - Run SparkPi with env and mount secrets. - Run PySpark on simple pi.py example - Run PySpark with Python3 to test a pyfiles example - Run PySpark with memory customization - Run in client mode. - Start pod creation from template - PVs with local storage - Launcher client dependencies - Test basic decommissioning - Run SparkR on simple dataframe.R example *** FAILED *** The code passed to eventually never returned normally. Attempted 190 times over 3.000988466737 minutes. Last failure message: false was not true. (KubernetesSuite.scala:386) Run completed in 14 minutes, 46 seconds. Total number of tests run: 19 Suites: completed 2, aborted 0 Tests: succeeded 18, failed 1, canceled 0, ignored 0, pending 0 *** 1 TEST FAILED *** ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on pull request #28594: [SPARK-31780][K8S][TESTS] Add R test tag to exclude R K8s image building and test
dongjoon-hyun edited a comment on pull request #28594: URL: https://github.com/apache/spark/pull/28594#issuecomment-663284883 BTW, I also have a personal Jenkins K8s machine on `master` branch and the following is the result last week (2020-07-16T12:14:18-07:00). Of course, I confirmed that `2020-07-20T22:25:51-07:00` test result is missing `Run SparkR on simple dataframe.R example` test. In short, I suspect that something happens on `master` branch between July 16 and July 20. ``` KubernetesSuite: - Run SparkPi with no resources - Run SparkPi with a very long application name. - Use SparkLauncher.NO_RESOURCE - Run SparkPi with a master URL without a scheme. - Run SparkPi with an argument. - Run SparkPi with custom labels, annotations, and environment variables. - All pods have the same service account by default - Run extraJVMOptions check on driver - Run SparkRemoteFileTest using a remote data file - Run SparkPi with env and mount secrets. - Run PySpark on simple pi.py example - Run PySpark with Python3 to test a pyfiles example - Run PySpark with memory customization - Run in client mode. - Start pod creation from template - PVs with local storage - Launcher client dependencies - Test basic decommissioning - Run SparkR on simple dataframe.R example Run completed in 9 minutes, 11 seconds. Total number of tests run: 19 Suites: completed 2, aborted 0 Tests: succeeded 19, failed 0, canceled 0, ignored 0, pending 0 All tests passed. [INFO] [INFO] Reactor Summary for Spark Project Parent POM 3.1.0-SNAPSHOT: [INFO] [INFO] Spark Project Parent POM ... SUCCESS [ 3.535 s] [INFO] Spark Project Tags . SUCCESS [ 9.053 s] [INFO] Spark Project Local DB . SUCCESS [ 5.057 s] [INFO] Spark Project Networking ... SUCCESS [ 6.522 s] [INFO] Spark Project Shuffle Streaming Service SUCCESS [ 4.582 s] [INFO] Spark Project Unsafe ... SUCCESS [ 10.181 s] [INFO] Spark Project Launcher . SUCCESS [ 4.874 s] [INFO] Spark Project Core . SUCCESS [02:05 min] [INFO] Spark Project Kubernetes Integration Tests . SUCCESS [12:29 min] [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 15:20 min [INFO] Finished at: 2020-07-16T12:14:18-07:00 [INFO] ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on pull request #28594: [SPARK-31780][K8S][TESTS] Add R test tag to exclude R K8s image building and test
dongjoon-hyun edited a comment on pull request #28594: URL: https://github.com/apache/spark/pull/28594#issuecomment-663281368 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on pull request #28594: [SPARK-31780][K8S][TESTS] Add R test tag to exclude R K8s image building and test
dongjoon-hyun edited a comment on pull request #28594: URL: https://github.com/apache/spark/pull/28594#issuecomment-663280946 @ifilonenko and @shaneknapp . Are you sure that is caused by this PR? > This has been a side-effect of this change. IIRC, at that time, this PR is tested correctly and I verified that R test are executed there. - https://github.com/apache/spark/pull/28594#issuecomment-631824311 I guess another PR after this commit may cause the missing R testing. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on pull request #28594: [SPARK-31780][K8S][TESTS] Add R test tag to exclude R K8s image building and test
dongjoon-hyun edited a comment on pull request #28594: URL: https://github.com/apache/spark/pull/28594#issuecomment-662839954 By default (=if you don't use `--exclude-tags r`), the test coverage is unchanged in the community so far. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
