[ https://issues.apache.org/jira/browse/SPARK-34131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17267653#comment-17267653 ]
Dongjoon Hyun commented on SPARK-34131: --------------------------------------- Hi, [~jlaskowski]. Does this bug exist in the latest version, too? Could you try the latest version like Apache Spark 3.1.1 RC1? > NPE when driver.podTemplateFile defines no containers > ----------------------------------------------------- > > Key: SPARK-34131 > URL: https://issues.apache.org/jira/browse/SPARK-34131 > Project: Spark > Issue Type: Bug > Components: Kubernetes > Affects Versions: 3.0.1 > Reporter: Jacek Laskowski > Priority: Minor > > An empty pod template leads to the following NPE: > {code} > 21/01/15 18:44:32 ERROR KubernetesUtils: Encountered exception while > attempting to load initial pod spec from file > java.lang.NullPointerException > at > org.apache.spark.deploy.k8s.KubernetesUtils$.selectSparkContainer(KubernetesUtils.scala:108) > at > org.apache.spark.deploy.k8s.KubernetesUtils$.loadPodFromTemplate(KubernetesUtils.scala:88) > at > org.apache.spark.deploy.k8s.submit.KubernetesDriverBuilder.$anonfun$buildFromFeatures$1(KubernetesDriverBuilder.scala:36) > at scala.Option.map(Option.scala:230) > at > org.apache.spark.deploy.k8s.submit.KubernetesDriverBuilder.buildFromFeatures(KubernetesDriverBuilder.scala:32) > at > org.apache.spark.deploy.k8s.submit.Client.run(KubernetesClientApplication.scala:98) > at > org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.$anonfun$run$4(KubernetesClientApplication.scala:221) > at > org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.$anonfun$run$4$adapted(KubernetesClientApplication.scala:215) > at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:2539) > at > org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.run(KubernetesClientApplication.scala:215) > at > org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.start(KubernetesClientApplication.scala:188) > at > org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:928) > at > org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180) > at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203) > at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90) > at > org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1007) > at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1016) > at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) > {code} > {code:java} > $> cat empty-template.yml > spec: > {code} > {code} > $> ./bin/run-example \ > --master k8s://$K8S_SERVER \ > --deploy-mode cluster \ > --conf spark.kubernetes.driver.podTemplateFile=empty-template.yml \ > --name $POD_NAME \ > --jars local:///opt/spark/examples/jars/spark-examples_2.12-3.0.1.jar \ > --conf spark.kubernetes.container.image=spark:v3.0.1 \ > --conf spark.kubernetes.driver.pod.name=$POD_NAME \ > --conf spark.kubernetes.namespace=spark-demo \ > --conf spark.kubernetes.authenticate.driver.serviceAccountName=spark \ > --verbose \ > SparkPi 10 > {code} > It appears that the implicit requirement is that there's at least one > well-defined container of any name (not necessarily > {{spark.kubernetes.driver.podTemplateContainerName}}). -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org