Hi Dawn,
Probably, you are providing the incorrect image(must be a java image) or the
incorrect master ip or the service account. Please verify the pod’s permissions
for the service account(‘spark’ in your case).
I have tried executing the same program as below:
./spark-submit --master k8s://https://<masterIP:port> --deploy-mode
cluster --name spark-pi --class org.apache.spark.examples.SparkPi
--conf spark.executor.instances=1 --conf
spark.kubernetes.container.image=<Image> --conf
spark.kubernetes.namespace=<namespace>
local:///opt/spark/examples/jars/spark-examples*.jar 5
And, I was able to see “Pi is roughly 3.139774279548559” in the pod’s output
log.
Hope this will help! 😊
Regards,
Breeta
From: dawn breaks <2005dawnbre...@gmail.com>
Sent: Wednesday, February 13, 2019 1:52 PM
To: user@spark.apache.org
Subject: Got fatal error when running spark 2.4.0 on k8s
we submit spark job to k8s by the following command, and the driver pod got an
error and exit. Anybody can help us to solve it?
./bin/spark-submit \
--master k8s://https://172.21.91.48:6443<https://172.21.91.48:6443/> \
--deploy-mode cluster \
--conf spark.kubernetes.authenticate.driver.serviceAccountName=spark \
--name spark-pi \
--class org.apache.spark.examples.SparkPi \
--conf spark.executor.instances=1 \
--conf spark.kubernetes.container.image=xxxRepo/spark:v2.4.0 \
local:///opt/spark/examples/jars/spark-examples*.jar \
5
The error detail info as following:
2019-02-13 07:13:06 ERROR SparkContext:91 - Error initializing SparkContext.
org.apache.spark.SparkException: External scheduler cannot be instantiated
at
org.apache.spark.SparkContext$.org$apache$spark$SparkContext$$createTaskScheduler(SparkContext.scala:2794)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:493)
at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2520)
at
org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:935)
at
org.apache.spark.sql.SparkSession$Builder$$anonfun$7.apply(SparkSession.scala:926)
at scala.Option.getOrElse(Option.scala:121)
at
org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:926)
at org.apache.spark.examples.SparkPi$.main(SparkPi.scala:31)
at org.apache.spark.examples.SparkPi.main(SparkPi.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at
org.apache.spark.deploy.SparkSubmit.org<http://org.apache.spark.deploy.sparksubmit.org/>$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:849)
at
org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:167)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:195)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
at
org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:924)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:933)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: io.fabric8.kubernetes.client.KubernetesClientException: An error has
occurred.
at
io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:62)
at
io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:53)
at
io.fabric8.kubernetes.client.utils.HttpClientUtils.createHttpClient(HttpClientUtils.java:167)
at
org.apache.spark.deploy.k8s.SparkKubernetesClientFactory$.createKubernetesClient(SparkKubernetesClientFactory.scala:84)
at
org.apache.spark.scheduler.cluster.k8s.KubernetesClusterManager.createSchedulerBackend(KubernetesClusterManager.scala:64)
at
org.apache.spark.SparkContext$.org$apache$spark$SparkContext$$createTaskScheduler(SparkContext.scala:2788)
... 20 more
Caused by: java.security.cert.CertificateException: Could not parse
certificate: java.io.IOException: Empty input
at
sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110)
at
java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339)
at
io.fabric8.kubernetes.client.internal.CertUtils.createTrustStore(CertUtils.java:93)
at
io.fabric8.kubernetes.client.internal.CertUtils.createTrustStore(CertUtils.java:71)
at
io.fabric8.kubernetes.client.internal.SSLUtils.trustManagers(SSLUtils.java:114)
at
io.fabric8.kubernetes.client.internal.SSLUtils.trustManagers(SSLUtils.java:93)
at
io.fabric8.kubernetes.client.utils.HttpClientUtils.createHttpClient(HttpClientUtils.java:63)
... 23 more
Caused by: java.io.IOException: Empty input
at
sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:106)
... 29 more
