TongWei1105 opened a new pull request, #60020: URL: https://github.com/apache/spark/pull/60020
### What changes were proposed in this pull request? Check the executor pod allocation limit before incrementing the executor ID counter. This allows exactly the configured number of creation attempts and prevents the counter from overflowing after reaching `Int.MaxValue`. Add regression coverage for limits of 1, 3, and `Int.MaxValue`, including subsequent allocation attempts after reaching the limit. JIRA: https://issues.apache.org/jira/browse/SPARK-60804 ### Why are the changes needed? The current implementation increments the counter and then rejects IDs greater than or equal to `spark.kubernetes.allocation.maximum`. A limit of N therefore permits only N-1 creation attempts. With the direct allocator and a limit of 1, the first allocation throws `Exceed the pod creation limit: 1` before creating any executor pod. ### Does this PR introduce _any_ user-facing change? Yes. The direct Kubernetes allocator now permits exactly N executor pod creation attempts when `spark.kubernetes.allocation.maximum=N`. Further attempts are still rejected. ### How was this patch tested? - `build/sbt -Pkubernetes 'kubernetes/testOnly *ExecutorPodsAllocatorSuite'`: all 32 tests passed. - `build/sbt -Pkubernetes 'kubernetes/scalastyle' 'kubernetes/Test/scalastyle'`: no errors or warnings. Tests verify the pod creation calls, the counter at the limit, and rejection of subsequent attempts without counter overflow. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: OpenAI Codex (tool version unavailable in this session). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
