ulysses-you opened a new pull request, #57329: URL: https://github.com/apache/spark/pull/57329
### What changes were proposed in this pull request? When a task fails with out-of-memory, the retry can now request more CPUs than the ResourceProfile's `spark.task.cpus`, controlled by a new config `spark.task.oomRetryCpusIncrement` (default 0 = disabled). Each OOM retry of a task gets `spark.task.cpus + spark.task.oomRetryCpusIncrement * N` CPUs (N = the number of OOM failures of that task), capped at the executor's total core count. Two OOM sources are recognized: - `SparkOutOfMemoryError` (execution memory pool exhaustion), via the `ExceptionFailure` path in `handleFailedTask`. - A JVM heap OutOfMemoryError that kills the executor, recognized by the executor exit code in `executorLost` (configurable via `spark.task.oomRetryExecutorExitCodes`, default 52). The extra CPUs reduce the number of concurrent tasks on the executor, which increases the retrying task's share of the execution memory pool (the pool is split by the number of active tasks). If the offered executor does not have enough free CPUs, the retry waits for a sufficient offer rather than falling back. Barrier stages are excluded. Speculative copies reuse the same per-index retry count, so they request the same CPUs as the running OOM retry. ### Why are the changes needed? Today an OOM-failed task is retried with identical resources, which is unlikely to succeed and wastes retry attempts. Giving the retry more CPUs (hence more memory) improves the chance of success. ### Does this PR introduce any user-facing change? Yes, two new configs `spark.task.oomRetryCpusIncrement` and `spark.task.oomRetryExecutorExitCodes`, both documented in configuration.md. Disabled by default (increment 0), so behavior is unchanged unless opted in. ### How was this patch tested? New unit tests in `TaskSetManagerSuite` and `TaskSchedulerImplSuite` covering the disabled default (zero regression), cpus increment, strict wait, no starvation of normal tasks, executor-cores cap, reset on success, both OOM recognition paths, configurable exit codes, barrier exclusion, speculative consistency, and the maxTaskFailures safety bound. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
