ashangit commented on PR #51096: URL: https://github.com/apache/spark/pull/51096#issuecomment-2948596562
Hi, Relying on pod template doesn't work when trying to modify the spark-kubernetes-driver/spark-kubernetes-executor as spark is adding it's generated pod config to the list of containers, see [spark/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesClientApplication.scala at 80657974ee691a36c5a761629260a65b2fb7fa03 · apache/spark](https://github.com/apache/spark/blob/80657974ee691a36c5a761629260a65b2fb7fa03/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesClientApplication.scala#L126) and [spark/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsAllocator.scala at 80657974ee691a36c5a761629260a65b2fb7fa03 · apache/spark](https://github.com/apache/spark/blob/80657974ee691a36c5a761629260a65b2fb7fa03/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsAllocator.scala#L441) Due to this it ends up by trying to submit a pod with 2 times the spark-kubernetes-* container. Ex. of error log thrown by the kube api: `Pod "test-nico-driver" is invalid: [spec.containers[1].image: Required value, spec.containers[2].name: Duplicate value: "spark-kubernetes-driver"]` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
