anshulbaliga7 opened a new pull request, #58807:
URL: https://github.com/apache/spark/pull/58807
### What changes were proposed in this pull request?
`ExecutorKubernetesCredentialsFeatureStep` now treats an explicitly empty
`spark.kubernetes.authenticate.executor.serviceAccountName` (and
`spark.kubernetes.authenticate.driver.serviceAccountName`) as unset. Both
optional configs are filtered with `.filter(_.nonEmpty)` before the fallback
chain runs, so:
- an empty executor account falls back to the driver account, as documented;
- if both are empty, the executor pod spec is left untouched instead of
receiving `""`.
The config description in `docs/running-on-kubernetes.md` is updated to say
"not set or is empty". The change is confined to the two config reads so it
stays independent of #58343 (SPARK-58910), which reworks the pod-template check
in the same step.
### Why are the changes needed?
The executor service account config is `createOptional`, so setting it to an
empty string yields `Some("")`.
`executorServiceAccount.orElse(driverServiceAccount)` then never falls back,
and `buildPodWithServiceAccount` writes an empty `serviceAccount` /
`serviceAccountName` into the executor pod. That contradicts the documented
behavior ("if this parameter is not setup, the fallback logic will use the
driver's service account") and produces a pod that names no usable
account. An empty value is easy to hit from templating or `--conf key=` in
submission scripts.
### Does this PR introduce _any_ user-facing change?
Yes, a bug fix. Before, `--conf
spark.kubernetes.authenticate.executor.serviceAccountName=` with a driver
account configured wrote `serviceAccountName: ""` to executor pods. Now such
executors use the driver's service account, matching the behavior when the
config is absent. Non-empty values behave exactly as before.
### How was this patch tested?
Two new tests in `ExecutorKubernetesCredentialsFeatureStepSuite`:
- empty executor account plus a driver account resolves to the driver
account;
- empty executor and driver accounts leave `serviceAccount` /
`serviceAccountName` null.
```
build/sbt -Pkubernetes "kubernetes/testOnly
org.apache.spark.deploy.k8s.features.ExecutorKubernetesCredentialsFeatureStepSuite"
```
All 5 tests pass (3 existing, 2 new).
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Fable 5.1)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
