Jacek Laskowski created SPARK-34131: ---------------------------------------
Summary: NPE when driver.podTemplateFile defines no containers Key: SPARK-34131 URL: https://issues.apache.org/jira/browse/SPARK-34131 Project: Spark Issue Type: Bug Components: Kubernetes Affects Versions: 3.0.1 Reporter: Jacek Laskowski An empty pod template leads to the following NPE: {code} 21/01/15 18:44:32 ERROR KubernetesUtils: Encountered exception while attempting to load initial pod spec from file java.lang.NullPointerException at org.apache.spark.deploy.k8s.KubernetesUtils$.selectSparkContainer(KubernetesUtils.scala:108) at org.apache.spark.deploy.k8s.KubernetesUtils$.loadPodFromTemplate(KubernetesUtils.scala:88) at org.apache.spark.deploy.k8s.submit.KubernetesDriverBuilder.$anonfun$buildFromFeatures$1(KubernetesDriverBuilder.scala:36) at scala.Option.map(Option.scala:230) at org.apache.spark.deploy.k8s.submit.KubernetesDriverBuilder.buildFromFeatures(KubernetesDriverBuilder.scala:32) at org.apache.spark.deploy.k8s.submit.Client.run(KubernetesClientApplication.scala:98) at org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.$anonfun$run$4(KubernetesClientApplication.scala:221) at org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.$anonfun$run$4$adapted(KubernetesClientApplication.scala:215) at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:2539) at org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.run(KubernetesClientApplication.scala:215) at org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.start(KubernetesClientApplication.scala:188) at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:928) at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180) at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203) at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90) at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1007) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1016) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) {code} {code:java} $> cat empty-template.yml spec: {code} {code} $> ./bin/run-example \ --master k8s://$K8S_SERVER \ --deploy-mode cluster \ --conf spark.kubernetes.driver.podTemplateFile=empty-template.yml \ --name $POD_NAME \ --jars local:///opt/spark/examples/jars/spark-examples_2.12-3.0.1.jar \ --conf spark.kubernetes.container.image=spark:v3.0.1 \ --conf spark.kubernetes.driver.pod.name=$POD_NAME \ --conf spark.kubernetes.namespace=spark-demo \ --conf spark.kubernetes.authenticate.driver.serviceAccountName=spark \ --verbose \ SparkPi 10 {code} It appears that the implicit requirement is that there's at least one well-defined container of any name (not necessarily {{spark.kubernetes.driver.podTemplateContainerName}}). -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org