[
https://issues.apache.org/jira/browse/SPARK-34131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17266760#comment-17266760
]
Hyukjin Kwon commented on SPARK-34131:
--------------------------------------
cc [~holdenkarau] and [~dongjoon] FYI
> NPE when driver.podTemplateFile defines no containers
> -----------------------------------------------------
>
> Key: SPARK-34131
> URL: https://issues.apache.org/jira/browse/SPARK-34131
> Project: Spark
> Issue Type: Bug
> Components: Kubernetes
> Affects Versions: 3.0.1
> Reporter: Jacek Laskowski
> Priority: Minor
>
> An empty pod template leads to the following NPE:
> {code}
> 21/01/15 18:44:32 ERROR KubernetesUtils: Encountered exception while
> attempting to load initial pod spec from file
> java.lang.NullPointerException
> at
> org.apache.spark.deploy.k8s.KubernetesUtils$.selectSparkContainer(KubernetesUtils.scala:108)
> at
> org.apache.spark.deploy.k8s.KubernetesUtils$.loadPodFromTemplate(KubernetesUtils.scala:88)
> at
> org.apache.spark.deploy.k8s.submit.KubernetesDriverBuilder.$anonfun$buildFromFeatures$1(KubernetesDriverBuilder.scala:36)
> at scala.Option.map(Option.scala:230)
> at
> org.apache.spark.deploy.k8s.submit.KubernetesDriverBuilder.buildFromFeatures(KubernetesDriverBuilder.scala:32)
> at
> org.apache.spark.deploy.k8s.submit.Client.run(KubernetesClientApplication.scala:98)
> at
> org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.$anonfun$run$4(KubernetesClientApplication.scala:221)
> at
> org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.$anonfun$run$4$adapted(KubernetesClientApplication.scala:215)
> at org.apache.spark.util.Utils$.tryWithResource(Utils.scala:2539)
> at
> org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.run(KubernetesClientApplication.scala:215)
> at
> org.apache.spark.deploy.k8s.submit.KubernetesClientApplication.start(KubernetesClientApplication.scala:188)
> at
> org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:928)
> at
> org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
> at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
> at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
> at
> org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1007)
> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1016)
> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
> {code}
> {code:java}
> $> cat empty-template.yml
> spec:
> {code}
> {code}
> $> ./bin/run-example \
> --master k8s://$K8S_SERVER \
> --deploy-mode cluster \
> --conf spark.kubernetes.driver.podTemplateFile=empty-template.yml \
> --name $POD_NAME \
> --jars local:///opt/spark/examples/jars/spark-examples_2.12-3.0.1.jar \
> --conf spark.kubernetes.container.image=spark:v3.0.1 \
> --conf spark.kubernetes.driver.pod.name=$POD_NAME \
> --conf spark.kubernetes.namespace=spark-demo \
> --conf spark.kubernetes.authenticate.driver.serviceAccountName=spark \
> --verbose \
> SparkPi 10
> {code}
> It appears that the implicit requirement is that there's at least one
> well-defined container of any name (not necessarily
> {{spark.kubernetes.driver.podTemplateContainerName}}).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
