It seems that fabric8 kubernetes client can't parse the caCertFile in the default location /var/run/secrets/kubernetes.io/serviceaccount/ca.crt, and anybody give me some advices?
On Wed, 13 Feb 2019 at 16:21, dawn breaks <2005dawnbre...@gmail.com> wrote: > we submit spark job to k8s by the following command, and the driver pod > got an error and exit. Anybody can help us to solve it? > > ./bin/spark-submit \ > --master k8s://https://172.21.91.48:6443 \ > --deploy-mode cluster \ > --conf spark.kubernetes.authenticate.driver.serviceAccountName=spark \ > --name spark-pi \ > --class org.apache.spark.examples.SparkPi \ > --conf spark.executor.instances=1 \ > --conf spark.kubernetes.container.image=xxxRepo/spark:v2.4.0 \ > local:///opt/spark/examples/jars/spark-examples*.jar \ > 5 > > > The error detail info as following: > > 2019-02-13 07:13:06 ERROR SparkContext:91 - Error initializing > SparkContext. > org.apache.spark.SparkException: External scheduler cannot be instantiated > at > org.apache.spark.SparkContext$.org$apache$spark$SparkContext$$createTaskScheduler(SparkContext.scala:2794) > at org.apache.spark.SparkContext.<init>(SparkContext.scala:493) > at > org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2520) > at > org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:935) > at > org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:926) > at scala.Option.getOrElse(Option.scala:121) > at > org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:926) > at org.apache.spark.examples.SparkPi$.main(SparkPi.scala:31) > at org.apache.spark.examples.SparkPi.main(SparkPi.scala) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52) > at org.apache.spark.deploy.SparkSubmit.org > <http://org.apache.spark.deploy.sparksubmit.org/> > $apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:849) > at > org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:167) > at > org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:195) > at > org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86) > at > org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:924) > at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:933) > at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) > Caused by: io.fabric8.kubernetes.client.KubernetesClientException: An > error has occurred. > at > io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:62) > at > io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:53) > at > io.fabric8.kubernetes.client.utils.HttpClientUtils.createHttpClient(HttpClientUtils.java:167) > at > org.apache.spark.deploy.k8s.SparkKubernetesClientFactory$.createKubernetesClient(SparkKubernetesClientFactory.scala:84) > at > org.apache.spark.scheduler.cluster.k8s.KubernetesClusterManager.createSchedulerBackend(KubernetesClusterManager.scala:64) > at > org.apache.spark.SparkContext$.org$apache$spark$SparkContext$$createTaskScheduler(SparkContext.scala:2788) > ... 20 more > Caused by: java.security.cert.CertificateException: Could not parse > certificate: java.io.IOException: Empty input > at > sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at > java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at > io.fabric8.kubernetes.client.internal.CertUtils.createTrustStore(CertUtils.java:93) > at > io.fabric8.kubernetes.client.internal.CertUtils.createTrustStore(CertUtils.java:71) > at > io.fabric8.kubernetes.client.internal.SSLUtils.trustManagers(SSLUtils.java:114) > at > io.fabric8.kubernetes.client.internal.SSLUtils.trustManagers(SSLUtils.java:93) > at > io.fabric8.kubernetes.client.utils.HttpClientUtils.createHttpClient(HttpClientUtils.java:63) > ... 23 more > Caused by: java.io.IOException: Empty input > at > sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:106) > ... 29 more >
