ulysses-you commented on code in PR #57329:
URL: https://github.com/apache/spark/pull/57329#discussion_r3614434461
##########
core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala:
##########
@@ -1247,6 +1318,11 @@ private[spark] class TaskSetManager(
// that the task is not running, and it is NetworkFailure rather
than TaskFailure.
case _ => !info.launching
}
+ // Grow the CPUs of the retry when a running task's executor died of a
JVM heap OOM. See
+ // the SparkOutOfMemoryError branch in handleFailedTask for the
non-fatal counterpart.
+ if (oomRetryCpusIncrement > 0 && !isBarrier && isOomExit &&
exitCausedByApp) {
Review Comment:
Fixed in 8f657a78422. The `ExceptionFailure` path now recognizes any
`OutOfMemoryError`, not only `SparkOutOfMemoryError`, via a helper:
```scala
private def isOom(ef: ExceptionFailure): Boolean = {
ef.exception.exists(_.isInstanceOf[OutOfMemoryError]) ||
ef.className == classOf[OutOfMemoryError].getName ||
ef.className == classOf[SparkOutOfMemoryError].getName
}
```
It prefers the preserved `Throwable` and falls back to the class name, since
the exception may not be preserved (`preserveCause = false`) or loadable in the
driver. So a fatal `java.lang.OutOfMemoryError` reported through the
`ExceptionFailure` (which the `Executor` sends before it exits with code 52)
now grows the retry's cpus even when the FAILED update is processed before the
executor-loss event. Double counting is avoided in both orderings: whichever
event arrives first does the increment, and the `info.finished` guard at the
top of `handleFailedTask` makes the second a no-op. Added two tests covering
FAILED-before-executor-loss and the reverse ordering, each asserting the
counter increments exactly once.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
