[jira] [Commented] (SPARK-32617) Upgrade kubernetes client version to support latest minikube version.
[ https://issues.apache.org/jira/browse/SPARK-32617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17291326#comment-17291326 ] Dongjoon Hyun commented on SPARK-32617: --- BTW, the issue title is wrong. K8s IT works with all the latest Minikube including v1.16 and v1.17 (as of today). > Upgrade kubernetes client version to support latest minikube version. > - > > Key: SPARK-32617 > URL: https://issues.apache.org/jira/browse/SPARK-32617 > Project: Spark > Issue Type: Bug > Components: Kubernetes >Affects Versions: 3.1.0 >Reporter: Prashant Sharma >Priority: Major > > Following error comes, when the k8s integration tests are run against the > minikube cluster with version 1.2.1 > {code:java} > Run starting. Expected test count is: 18 > KubernetesSuite: > org.apache.spark.deploy.k8s.integrationtest.KubernetesSuite *** ABORTED *** > io.fabric8.kubernetes.client.KubernetesClientException: An error has > occurred. > at > io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:64) > at > io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:53) > at > io.fabric8.kubernetes.client.utils.HttpClientUtils.createHttpClient(HttpClientUtils.java:196) > at > io.fabric8.kubernetes.client.utils.HttpClientUtils.createHttpClient(HttpClientUtils.java:62) > at io.fabric8.kubernetes.client.BaseClient.(BaseClient.java:51) > at > io.fabric8.kubernetes.client.DefaultKubernetesClient.(DefaultKubernetesClient.java:105) > at > org.apache.spark.deploy.k8s.integrationtest.backend.minikube.Minikube$.getKubernetesClient(Minikube.scala:81) > at > org.apache.spark.deploy.k8s.integrationtest.backend.minikube.MinikubeTestBackend$.initialize(MinikubeTestBackend.scala:33) > at > org.apache.spark.deploy.k8s.integrationtest.KubernetesSuite.beforeAll(KubernetesSuite.scala:131) > at > org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212) > ... > Cause: java.nio.file.NoSuchFileException: /root/.minikube/apiserver.crt > at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) > at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) > at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) > at > sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214) > at java.nio.file.Files.newByteChannel(Files.java:361) > at java.nio.file.Files.newByteChannel(Files.java:407) > at java.nio.file.Files.readAllBytes(Files.java:3152) > at > io.fabric8.kubernetes.client.internal.CertUtils.getInputStreamFromDataOrFile(CertUtils.java:72) > at > io.fabric8.kubernetes.client.internal.CertUtils.createKeyStore(CertUtils.java:242) > at > io.fabric8.kubernetes.client.internal.SSLUtils.keyManagers(SSLUtils.java:128) > ... > Run completed in 1 second, 821 milliseconds. > Total number of tests run: 0 > Suites: completed 1, aborted 1 > Tests: succeeded 0, failed 0, canceled 0, ignored 0, pending 0 > *** 1 SUITE ABORTED *** > [INFO] > > [INFO] Reactor Summary for Spark Project Parent POM 3.1.0-SNAPSHOT: > [INFO] > [INFO] Spark Project Parent POM ... SUCCESS [ 4.454 > s] > [INFO] Spark Project Tags . SUCCESS [ 4.768 > s] > [INFO] Spark Project Local DB . SUCCESS [ 2.961 > s] > [INFO] Spark Project Networking ... SUCCESS [ 4.258 > s] > [INFO] Spark Project Shuffle Streaming Service SUCCESS [ 5.703 > s] > [INFO] Spark Project Unsafe ... SUCCESS [ 3.239 > s] > [INFO] Spark Project Launcher . SUCCESS [ 3.224 > s] > [INFO] Spark Project Core . SUCCESS [02:25 > min] > [INFO] Spark Project Kubernetes Integration Tests . FAILURE [ 17.244 > s] > [INFO] > > [INFO] BUILD FAILURE > [INFO] > > [INFO] Total time: 03:12 min > [INFO] Finished at: 2020-08-11T06:26:15-05:00 > [INFO] > > [ERROR] Failed to execute goal > org.scalatest:scalatest-maven-plugin:2.0.0:test (integration-test) on project > spark-kubernetes-integration-tests_2.12: There are test failures -> [Help 1] > [ERROR] > [ERROR] To see the full stack trace of the errors, re-run Maven with the -e > switch. > [ERROR] Re-run Maven using the -X switch to enable full debug logging. > [ERROR] > [ERROR] For more information
[jira] [Commented] (SPARK-32617) Upgrade kubernetes client version to support latest minikube version.
[ https://issues.apache.org/jira/browse/SPARK-32617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17291325#comment-17291325 ] Dongjoon Hyun commented on SPARK-32617: --- The commit is reverted because it breaks IT when `minikube kubectl config view` returns multiple configs. I tried to do follow-up first with `kubectl config view --minify`. However, it turns out that `minikube` doesn't support `minikube kubectl config view --minify`. > Upgrade kubernetes client version to support latest minikube version. > - > > Key: SPARK-32617 > URL: https://issues.apache.org/jira/browse/SPARK-32617 > Project: Spark > Issue Type: Bug > Components: Kubernetes >Affects Versions: 3.1.0 >Reporter: Prashant Sharma >Priority: Major > > Following error comes, when the k8s integration tests are run against the > minikube cluster with version 1.2.1 > {code:java} > Run starting. Expected test count is: 18 > KubernetesSuite: > org.apache.spark.deploy.k8s.integrationtest.KubernetesSuite *** ABORTED *** > io.fabric8.kubernetes.client.KubernetesClientException: An error has > occurred. > at > io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:64) > at > io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:53) > at > io.fabric8.kubernetes.client.utils.HttpClientUtils.createHttpClient(HttpClientUtils.java:196) > at > io.fabric8.kubernetes.client.utils.HttpClientUtils.createHttpClient(HttpClientUtils.java:62) > at io.fabric8.kubernetes.client.BaseClient.(BaseClient.java:51) > at > io.fabric8.kubernetes.client.DefaultKubernetesClient.(DefaultKubernetesClient.java:105) > at > org.apache.spark.deploy.k8s.integrationtest.backend.minikube.Minikube$.getKubernetesClient(Minikube.scala:81) > at > org.apache.spark.deploy.k8s.integrationtest.backend.minikube.MinikubeTestBackend$.initialize(MinikubeTestBackend.scala:33) > at > org.apache.spark.deploy.k8s.integrationtest.KubernetesSuite.beforeAll(KubernetesSuite.scala:131) > at > org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212) > ... > Cause: java.nio.file.NoSuchFileException: /root/.minikube/apiserver.crt > at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) > at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) > at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) > at > sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214) > at java.nio.file.Files.newByteChannel(Files.java:361) > at java.nio.file.Files.newByteChannel(Files.java:407) > at java.nio.file.Files.readAllBytes(Files.java:3152) > at > io.fabric8.kubernetes.client.internal.CertUtils.getInputStreamFromDataOrFile(CertUtils.java:72) > at > io.fabric8.kubernetes.client.internal.CertUtils.createKeyStore(CertUtils.java:242) > at > io.fabric8.kubernetes.client.internal.SSLUtils.keyManagers(SSLUtils.java:128) > ... > Run completed in 1 second, 821 milliseconds. > Total number of tests run: 0 > Suites: completed 1, aborted 1 > Tests: succeeded 0, failed 0, canceled 0, ignored 0, pending 0 > *** 1 SUITE ABORTED *** > [INFO] > > [INFO] Reactor Summary for Spark Project Parent POM 3.1.0-SNAPSHOT: > [INFO] > [INFO] Spark Project Parent POM ... SUCCESS [ 4.454 > s] > [INFO] Spark Project Tags . SUCCESS [ 4.768 > s] > [INFO] Spark Project Local DB . SUCCESS [ 2.961 > s] > [INFO] Spark Project Networking ... SUCCESS [ 4.258 > s] > [INFO] Spark Project Shuffle Streaming Service SUCCESS [ 5.703 > s] > [INFO] Spark Project Unsafe ... SUCCESS [ 3.239 > s] > [INFO] Spark Project Launcher . SUCCESS [ 3.224 > s] > [INFO] Spark Project Core . SUCCESS [02:25 > min] > [INFO] Spark Project Kubernetes Integration Tests . FAILURE [ 17.244 > s] > [INFO] > > [INFO] BUILD FAILURE > [INFO] > > [INFO] Total time: 03:12 min > [INFO] Finished at: 2020-08-11T06:26:15-05:00 > [INFO] > > [ERROR] Failed to execute goal > org.scalatest:scalatest-maven-plugin:2.0.0:test (integration-test) on project > spark-kubernetes-integration-tests_2.12: There are test failures -> [Help 1] > [ERROR] > [ERROR] To see the full stack trace of the er
[jira] [Commented] (SPARK-32617) Upgrade kubernetes client version to support latest minikube version.
[ https://issues.apache.org/jira/browse/SPARK-32617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17291321#comment-17291321 ] Apache Spark commented on SPARK-32617: -- User 'dongjoon-hyun' has created a pull request for this issue: https://github.com/apache/spark/pull/31655 > Upgrade kubernetes client version to support latest minikube version. > - > > Key: SPARK-32617 > URL: https://issues.apache.org/jira/browse/SPARK-32617 > Project: Spark > Issue Type: Bug > Components: Kubernetes >Affects Versions: 3.1.0 >Reporter: Prashant Sharma >Assignee: Attila Zsolt Piros >Priority: Major > Fix For: 3.2.0 > > > Following error comes, when the k8s integration tests are run against the > minikube cluster with version 1.2.1 > {code:java} > Run starting. Expected test count is: 18 > KubernetesSuite: > org.apache.spark.deploy.k8s.integrationtest.KubernetesSuite *** ABORTED *** > io.fabric8.kubernetes.client.KubernetesClientException: An error has > occurred. > at > io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:64) > at > io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:53) > at > io.fabric8.kubernetes.client.utils.HttpClientUtils.createHttpClient(HttpClientUtils.java:196) > at > io.fabric8.kubernetes.client.utils.HttpClientUtils.createHttpClient(HttpClientUtils.java:62) > at io.fabric8.kubernetes.client.BaseClient.(BaseClient.java:51) > at > io.fabric8.kubernetes.client.DefaultKubernetesClient.(DefaultKubernetesClient.java:105) > at > org.apache.spark.deploy.k8s.integrationtest.backend.minikube.Minikube$.getKubernetesClient(Minikube.scala:81) > at > org.apache.spark.deploy.k8s.integrationtest.backend.minikube.MinikubeTestBackend$.initialize(MinikubeTestBackend.scala:33) > at > org.apache.spark.deploy.k8s.integrationtest.KubernetesSuite.beforeAll(KubernetesSuite.scala:131) > at > org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212) > ... > Cause: java.nio.file.NoSuchFileException: /root/.minikube/apiserver.crt > at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) > at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) > at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) > at > sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214) > at java.nio.file.Files.newByteChannel(Files.java:361) > at java.nio.file.Files.newByteChannel(Files.java:407) > at java.nio.file.Files.readAllBytes(Files.java:3152) > at > io.fabric8.kubernetes.client.internal.CertUtils.getInputStreamFromDataOrFile(CertUtils.java:72) > at > io.fabric8.kubernetes.client.internal.CertUtils.createKeyStore(CertUtils.java:242) > at > io.fabric8.kubernetes.client.internal.SSLUtils.keyManagers(SSLUtils.java:128) > ... > Run completed in 1 second, 821 milliseconds. > Total number of tests run: 0 > Suites: completed 1, aborted 1 > Tests: succeeded 0, failed 0, canceled 0, ignored 0, pending 0 > *** 1 SUITE ABORTED *** > [INFO] > > [INFO] Reactor Summary for Spark Project Parent POM 3.1.0-SNAPSHOT: > [INFO] > [INFO] Spark Project Parent POM ... SUCCESS [ 4.454 > s] > [INFO] Spark Project Tags . SUCCESS [ 4.768 > s] > [INFO] Spark Project Local DB . SUCCESS [ 2.961 > s] > [INFO] Spark Project Networking ... SUCCESS [ 4.258 > s] > [INFO] Spark Project Shuffle Streaming Service SUCCESS [ 5.703 > s] > [INFO] Spark Project Unsafe ... SUCCESS [ 3.239 > s] > [INFO] Spark Project Launcher . SUCCESS [ 3.224 > s] > [INFO] Spark Project Core . SUCCESS [02:25 > min] > [INFO] Spark Project Kubernetes Integration Tests . FAILURE [ 17.244 > s] > [INFO] > > [INFO] BUILD FAILURE > [INFO] > > [INFO] Total time: 03:12 min > [INFO] Finished at: 2020-08-11T06:26:15-05:00 > [INFO] > > [ERROR] Failed to execute goal > org.scalatest:scalatest-maven-plugin:2.0.0:test (integration-test) on project > spark-kubernetes-integration-tests_2.12: There are test failures -> [Help 1] > [ERROR] > [ERROR] To see the full stack trace of the errors, re-run Maven with the -e > switch. > [ERROR] Re-run Maven using the -X switch to enable
[jira] [Commented] (SPARK-32617) Upgrade kubernetes client version to support latest minikube version.
[ https://issues.apache.org/jira/browse/SPARK-32617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17248523#comment-17248523 ] Apache Spark commented on SPARK-32617: -- User 'attilapiros' has created a pull request for this issue: https://github.com/apache/spark/pull/30751 > Upgrade kubernetes client version to support latest minikube version. > - > > Key: SPARK-32617 > URL: https://issues.apache.org/jira/browse/SPARK-32617 > Project: Spark > Issue Type: Bug > Components: Kubernetes >Affects Versions: 3.1.0 >Reporter: Prashant Sharma >Priority: Major > > Following error comes, when the k8s integration tests are run against the > minikube cluster with version 1.2.1 > {code:java} > Run starting. Expected test count is: 18 > KubernetesSuite: > org.apache.spark.deploy.k8s.integrationtest.KubernetesSuite *** ABORTED *** > io.fabric8.kubernetes.client.KubernetesClientException: An error has > occurred. > at > io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:64) > at > io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:53) > at > io.fabric8.kubernetes.client.utils.HttpClientUtils.createHttpClient(HttpClientUtils.java:196) > at > io.fabric8.kubernetes.client.utils.HttpClientUtils.createHttpClient(HttpClientUtils.java:62) > at io.fabric8.kubernetes.client.BaseClient.(BaseClient.java:51) > at > io.fabric8.kubernetes.client.DefaultKubernetesClient.(DefaultKubernetesClient.java:105) > at > org.apache.spark.deploy.k8s.integrationtest.backend.minikube.Minikube$.getKubernetesClient(Minikube.scala:81) > at > org.apache.spark.deploy.k8s.integrationtest.backend.minikube.MinikubeTestBackend$.initialize(MinikubeTestBackend.scala:33) > at > org.apache.spark.deploy.k8s.integrationtest.KubernetesSuite.beforeAll(KubernetesSuite.scala:131) > at > org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212) > ... > Cause: java.nio.file.NoSuchFileException: /root/.minikube/apiserver.crt > at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) > at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) > at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) > at > sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214) > at java.nio.file.Files.newByteChannel(Files.java:361) > at java.nio.file.Files.newByteChannel(Files.java:407) > at java.nio.file.Files.readAllBytes(Files.java:3152) > at > io.fabric8.kubernetes.client.internal.CertUtils.getInputStreamFromDataOrFile(CertUtils.java:72) > at > io.fabric8.kubernetes.client.internal.CertUtils.createKeyStore(CertUtils.java:242) > at > io.fabric8.kubernetes.client.internal.SSLUtils.keyManagers(SSLUtils.java:128) > ... > Run completed in 1 second, 821 milliseconds. > Total number of tests run: 0 > Suites: completed 1, aborted 1 > Tests: succeeded 0, failed 0, canceled 0, ignored 0, pending 0 > *** 1 SUITE ABORTED *** > [INFO] > > [INFO] Reactor Summary for Spark Project Parent POM 3.1.0-SNAPSHOT: > [INFO] > [INFO] Spark Project Parent POM ... SUCCESS [ 4.454 > s] > [INFO] Spark Project Tags . SUCCESS [ 4.768 > s] > [INFO] Spark Project Local DB . SUCCESS [ 2.961 > s] > [INFO] Spark Project Networking ... SUCCESS [ 4.258 > s] > [INFO] Spark Project Shuffle Streaming Service SUCCESS [ 5.703 > s] > [INFO] Spark Project Unsafe ... SUCCESS [ 3.239 > s] > [INFO] Spark Project Launcher . SUCCESS [ 3.224 > s] > [INFO] Spark Project Core . SUCCESS [02:25 > min] > [INFO] Spark Project Kubernetes Integration Tests . FAILURE [ 17.244 > s] > [INFO] > > [INFO] BUILD FAILURE > [INFO] > > [INFO] Total time: 03:12 min > [INFO] Finished at: 2020-08-11T06:26:15-05:00 > [INFO] > > [ERROR] Failed to execute goal > org.scalatest:scalatest-maven-plugin:2.0.0:test (integration-test) on project > spark-kubernetes-integration-tests_2.12: There are test failures -> [Help 1] > [ERROR] > [ERROR] To see the full stack trace of the errors, re-run Maven with the -e > switch. > [ERROR] Re-run Maven using the -X switch to enable full debug logging. > [ERROR] > [ERROR] For more information about the er
[jira] [Commented] (SPARK-32617) Upgrade kubernetes client version to support latest minikube version.
[ https://issues.apache.org/jira/browse/SPARK-32617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17248522#comment-17248522 ] Apache Spark commented on SPARK-32617: -- User 'attilapiros' has created a pull request for this issue: https://github.com/apache/spark/pull/30751 > Upgrade kubernetes client version to support latest minikube version. > - > > Key: SPARK-32617 > URL: https://issues.apache.org/jira/browse/SPARK-32617 > Project: Spark > Issue Type: Bug > Components: Kubernetes >Affects Versions: 3.1.0 >Reporter: Prashant Sharma >Priority: Major > > Following error comes, when the k8s integration tests are run against the > minikube cluster with version 1.2.1 > {code:java} > Run starting. Expected test count is: 18 > KubernetesSuite: > org.apache.spark.deploy.k8s.integrationtest.KubernetesSuite *** ABORTED *** > io.fabric8.kubernetes.client.KubernetesClientException: An error has > occurred. > at > io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:64) > at > io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:53) > at > io.fabric8.kubernetes.client.utils.HttpClientUtils.createHttpClient(HttpClientUtils.java:196) > at > io.fabric8.kubernetes.client.utils.HttpClientUtils.createHttpClient(HttpClientUtils.java:62) > at io.fabric8.kubernetes.client.BaseClient.(BaseClient.java:51) > at > io.fabric8.kubernetes.client.DefaultKubernetesClient.(DefaultKubernetesClient.java:105) > at > org.apache.spark.deploy.k8s.integrationtest.backend.minikube.Minikube$.getKubernetesClient(Minikube.scala:81) > at > org.apache.spark.deploy.k8s.integrationtest.backend.minikube.MinikubeTestBackend$.initialize(MinikubeTestBackend.scala:33) > at > org.apache.spark.deploy.k8s.integrationtest.KubernetesSuite.beforeAll(KubernetesSuite.scala:131) > at > org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212) > ... > Cause: java.nio.file.NoSuchFileException: /root/.minikube/apiserver.crt > at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) > at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) > at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) > at > sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214) > at java.nio.file.Files.newByteChannel(Files.java:361) > at java.nio.file.Files.newByteChannel(Files.java:407) > at java.nio.file.Files.readAllBytes(Files.java:3152) > at > io.fabric8.kubernetes.client.internal.CertUtils.getInputStreamFromDataOrFile(CertUtils.java:72) > at > io.fabric8.kubernetes.client.internal.CertUtils.createKeyStore(CertUtils.java:242) > at > io.fabric8.kubernetes.client.internal.SSLUtils.keyManagers(SSLUtils.java:128) > ... > Run completed in 1 second, 821 milliseconds. > Total number of tests run: 0 > Suites: completed 1, aborted 1 > Tests: succeeded 0, failed 0, canceled 0, ignored 0, pending 0 > *** 1 SUITE ABORTED *** > [INFO] > > [INFO] Reactor Summary for Spark Project Parent POM 3.1.0-SNAPSHOT: > [INFO] > [INFO] Spark Project Parent POM ... SUCCESS [ 4.454 > s] > [INFO] Spark Project Tags . SUCCESS [ 4.768 > s] > [INFO] Spark Project Local DB . SUCCESS [ 2.961 > s] > [INFO] Spark Project Networking ... SUCCESS [ 4.258 > s] > [INFO] Spark Project Shuffle Streaming Service SUCCESS [ 5.703 > s] > [INFO] Spark Project Unsafe ... SUCCESS [ 3.239 > s] > [INFO] Spark Project Launcher . SUCCESS [ 3.224 > s] > [INFO] Spark Project Core . SUCCESS [02:25 > min] > [INFO] Spark Project Kubernetes Integration Tests . FAILURE [ 17.244 > s] > [INFO] > > [INFO] BUILD FAILURE > [INFO] > > [INFO] Total time: 03:12 min > [INFO] Finished at: 2020-08-11T06:26:15-05:00 > [INFO] > > [ERROR] Failed to execute goal > org.scalatest:scalatest-maven-plugin:2.0.0:test (integration-test) on project > spark-kubernetes-integration-tests_2.12: There are test failures -> [Help 1] > [ERROR] > [ERROR] To see the full stack trace of the errors, re-run Maven with the -e > switch. > [ERROR] Re-run Maven using the -X switch to enable full debug logging. > [ERROR] > [ERROR] For more information about the er
[jira] [Commented] (SPARK-32617) Upgrade kubernetes client version to support latest minikube version.
[ https://issues.apache.org/jira/browse/SPARK-32617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17248168#comment-17248168 ] Attila Zsolt Piros commented on SPARK-32617: I will have PR for this soon. > Upgrade kubernetes client version to support latest minikube version. > - > > Key: SPARK-32617 > URL: https://issues.apache.org/jira/browse/SPARK-32617 > Project: Spark > Issue Type: Bug > Components: Kubernetes >Affects Versions: 3.1.0 >Reporter: Prashant Sharma >Priority: Major > > Following error comes, when the k8s integration tests are run against the > minikube cluster with version 1.2.1 > {code:java} > Run starting. Expected test count is: 18 > KubernetesSuite: > org.apache.spark.deploy.k8s.integrationtest.KubernetesSuite *** ABORTED *** > io.fabric8.kubernetes.client.KubernetesClientException: An error has > occurred. > at > io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:64) > at > io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:53) > at > io.fabric8.kubernetes.client.utils.HttpClientUtils.createHttpClient(HttpClientUtils.java:196) > at > io.fabric8.kubernetes.client.utils.HttpClientUtils.createHttpClient(HttpClientUtils.java:62) > at io.fabric8.kubernetes.client.BaseClient.(BaseClient.java:51) > at > io.fabric8.kubernetes.client.DefaultKubernetesClient.(DefaultKubernetesClient.java:105) > at > org.apache.spark.deploy.k8s.integrationtest.backend.minikube.Minikube$.getKubernetesClient(Minikube.scala:81) > at > org.apache.spark.deploy.k8s.integrationtest.backend.minikube.MinikubeTestBackend$.initialize(MinikubeTestBackend.scala:33) > at > org.apache.spark.deploy.k8s.integrationtest.KubernetesSuite.beforeAll(KubernetesSuite.scala:131) > at > org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212) > ... > Cause: java.nio.file.NoSuchFileException: /root/.minikube/apiserver.crt > at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) > at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) > at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) > at > sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214) > at java.nio.file.Files.newByteChannel(Files.java:361) > at java.nio.file.Files.newByteChannel(Files.java:407) > at java.nio.file.Files.readAllBytes(Files.java:3152) > at > io.fabric8.kubernetes.client.internal.CertUtils.getInputStreamFromDataOrFile(CertUtils.java:72) > at > io.fabric8.kubernetes.client.internal.CertUtils.createKeyStore(CertUtils.java:242) > at > io.fabric8.kubernetes.client.internal.SSLUtils.keyManagers(SSLUtils.java:128) > ... > Run completed in 1 second, 821 milliseconds. > Total number of tests run: 0 > Suites: completed 1, aborted 1 > Tests: succeeded 0, failed 0, canceled 0, ignored 0, pending 0 > *** 1 SUITE ABORTED *** > [INFO] > > [INFO] Reactor Summary for Spark Project Parent POM 3.1.0-SNAPSHOT: > [INFO] > [INFO] Spark Project Parent POM ... SUCCESS [ 4.454 > s] > [INFO] Spark Project Tags . SUCCESS [ 4.768 > s] > [INFO] Spark Project Local DB . SUCCESS [ 2.961 > s] > [INFO] Spark Project Networking ... SUCCESS [ 4.258 > s] > [INFO] Spark Project Shuffle Streaming Service SUCCESS [ 5.703 > s] > [INFO] Spark Project Unsafe ... SUCCESS [ 3.239 > s] > [INFO] Spark Project Launcher . SUCCESS [ 3.224 > s] > [INFO] Spark Project Core . SUCCESS [02:25 > min] > [INFO] Spark Project Kubernetes Integration Tests . FAILURE [ 17.244 > s] > [INFO] > > [INFO] BUILD FAILURE > [INFO] > > [INFO] Total time: 03:12 min > [INFO] Finished at: 2020-08-11T06:26:15-05:00 > [INFO] > > [ERROR] Failed to execute goal > org.scalatest:scalatest-maven-plugin:2.0.0:test (integration-test) on project > spark-kubernetes-integration-tests_2.12: There are test failures -> [Help 1] > [ERROR] > [ERROR] To see the full stack trace of the errors, re-run Maven with the -e > switch. > [ERROR] Re-run Maven using the -X switch to enable full debug logging. > [ERROR] > [ERROR] For more information about the errors and possible solutions, please > read the following arti
[jira] [Commented] (SPARK-32617) Upgrade kubernetes client version to support latest minikube version.
[ https://issues.apache.org/jira/browse/SPARK-32617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17182945#comment-17182945 ] Daniel Moore commented on SPARK-32617: -- Is there a minikube support matrix? > Upgrade kubernetes client version to support latest minikube version. > - > > Key: SPARK-32617 > URL: https://issues.apache.org/jira/browse/SPARK-32617 > Project: Spark > Issue Type: Bug > Components: Kubernetes >Affects Versions: 3.1.0 >Reporter: Prashant Sharma >Priority: Major > > Following error comes, when the k8s integration tests are run against the > minikube cluster with version 1.2.1 > {code:java} > Run starting. Expected test count is: 18 > KubernetesSuite: > org.apache.spark.deploy.k8s.integrationtest.KubernetesSuite *** ABORTED *** > io.fabric8.kubernetes.client.KubernetesClientException: An error has > occurred. > at > io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:64) > at > io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:53) > at > io.fabric8.kubernetes.client.utils.HttpClientUtils.createHttpClient(HttpClientUtils.java:196) > at > io.fabric8.kubernetes.client.utils.HttpClientUtils.createHttpClient(HttpClientUtils.java:62) > at io.fabric8.kubernetes.client.BaseClient.(BaseClient.java:51) > at > io.fabric8.kubernetes.client.DefaultKubernetesClient.(DefaultKubernetesClient.java:105) > at > org.apache.spark.deploy.k8s.integrationtest.backend.minikube.Minikube$.getKubernetesClient(Minikube.scala:81) > at > org.apache.spark.deploy.k8s.integrationtest.backend.minikube.MinikubeTestBackend$.initialize(MinikubeTestBackend.scala:33) > at > org.apache.spark.deploy.k8s.integrationtest.KubernetesSuite.beforeAll(KubernetesSuite.scala:131) > at > org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212) > ... > Cause: java.nio.file.NoSuchFileException: /root/.minikube/apiserver.crt > at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) > at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) > at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) > at > sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214) > at java.nio.file.Files.newByteChannel(Files.java:361) > at java.nio.file.Files.newByteChannel(Files.java:407) > at java.nio.file.Files.readAllBytes(Files.java:3152) > at > io.fabric8.kubernetes.client.internal.CertUtils.getInputStreamFromDataOrFile(CertUtils.java:72) > at > io.fabric8.kubernetes.client.internal.CertUtils.createKeyStore(CertUtils.java:242) > at > io.fabric8.kubernetes.client.internal.SSLUtils.keyManagers(SSLUtils.java:128) > ... > Run completed in 1 second, 821 milliseconds. > Total number of tests run: 0 > Suites: completed 1, aborted 1 > Tests: succeeded 0, failed 0, canceled 0, ignored 0, pending 0 > *** 1 SUITE ABORTED *** > [INFO] > > [INFO] Reactor Summary for Spark Project Parent POM 3.1.0-SNAPSHOT: > [INFO] > [INFO] Spark Project Parent POM ... SUCCESS [ 4.454 > s] > [INFO] Spark Project Tags . SUCCESS [ 4.768 > s] > [INFO] Spark Project Local DB . SUCCESS [ 2.961 > s] > [INFO] Spark Project Networking ... SUCCESS [ 4.258 > s] > [INFO] Spark Project Shuffle Streaming Service SUCCESS [ 5.703 > s] > [INFO] Spark Project Unsafe ... SUCCESS [ 3.239 > s] > [INFO] Spark Project Launcher . SUCCESS [ 3.224 > s] > [INFO] Spark Project Core . SUCCESS [02:25 > min] > [INFO] Spark Project Kubernetes Integration Tests . FAILURE [ 17.244 > s] > [INFO] > > [INFO] BUILD FAILURE > [INFO] > > [INFO] Total time: 03:12 min > [INFO] Finished at: 2020-08-11T06:26:15-05:00 > [INFO] > > [ERROR] Failed to execute goal > org.scalatest:scalatest-maven-plugin:2.0.0:test (integration-test) on project > spark-kubernetes-integration-tests_2.12: There are test failures -> [Help 1] > [ERROR] > [ERROR] To see the full stack trace of the errors, re-run Maven with the -e > switch. > [ERROR] Re-run Maven using the -X switch to enable full debug logging. > [ERROR] > [ERROR] For more information about the errors and possible solutions, please > read the following articles: >