Jacek Laskowski created SPARK-34131:
---------------------------------------
Summary: NPE when driver.podTemplateFile defines no containers
Key: SPARK-34131
URL: https://issues.apache.org/jira/browse/SPARK-34131
Project: Spark
Issue Type: Bug
Components: Kubernetes
Affects Versions: 3.0.1
Reporter: Jacek Laskowski
An empty pod template leads to the following NPE:
{code}
21/01/15 18:44:32 ERROR KubernetesUtils: Encountered exception while attempting
to load initial pod spec from file
java.lang.NullPointerException
at
org.apache.spark.deploy.k8s.KubernetesUtils$.selectSparkContainer(KubernetesUtils.scala:108)
at
org.apache.spark.deploy.k8s.KubernetesUtils$.loadPodFromTemplate(KubernetesUtils.scala:88)
at
org.apache.spark.deploy.k8s.submit.KubernetesDriverBuilder.$anonfun$buildFromFeatures$1(KubernetesDriverBuilder.scala:36)
at scala.Option.map(Option.scala:230)
at
org.apache.spark.deploy.k8s.submit.KubernetesDriverBuilder.buildFromFeatures(KubernetesDriverBuilder.scala:32)
at
org.apache.spark.deploy.k8s.submit.Client.run(KubernetesClientApplication.scala:98)
at
org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.$anonfun$run$4(KubernetesClientApplication.scala:221)
at
org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.$anonfun$run$4$adapted(KubernetesClientApplication.scala:215)
at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:2539)
at
org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.run(KubernetesClientApplication.scala:215)
at
org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.start(KubernetesClientApplication.scala:188)
at
org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:928)
at
org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
at
org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1007)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1016)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
{code}
{code:java}
$> cat empty-template.yml
spec:
{code}
{code}
$> ./bin/run-example \
--master k8s://$K8S_SERVER \
--deploy-mode cluster \
--conf spark.kubernetes.driver.podTemplateFile=empty-template.yml \
--name $POD_NAME \
--jars local:///opt/spark/examples/jars/spark-examples_2.12-3.0.1.jar \
--conf spark.kubernetes.container.image=spark:v3.0.1 \
--conf spark.kubernetes.driver.pod.name=$POD_NAME \
--conf spark.kubernetes.namespace=spark-demo \
--conf spark.kubernetes.authenticate.driver.serviceAccountName=spark \
--verbose \
SparkPi 10
{code}
It appears that the implicit requirement is that there's at least one
well-defined container of any name (not necessarily
{{spark.kubernetes.driver.podTemplateContainerName}}).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
