Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/20408#discussion_r164886701
--- Diff:
core/src/main/scala/org/apache/spark/status/AppStatusListener.scala ---
@@ -228,9 +230,12 @@ private[spark] class AppStatusListener(
// Implicitly blacklist every available executor for the stage
associated with this node
Option(liveStages.get((event.stageId, event.stageAttemptId))).foreach
{ stage =>
- liveExecutors.values.filter(_.host == event.hostId).foreach { exec =>
- setStageBlackListStatus(stage, exec.executorId, now)
+ val executorIds = liveExecutors.values.filter(_.host ==
event.hostId).map(_.executorId)
+ executorIds.foreach { executorId =>
+ setStageBlackListStatus(stage, executorId, now)
}
+ stage.blackListedExecutors ++= executorIds
+ maybeUpdate(stage, now)
--- End diff --
if `setStageBlacklistStatus` took a Seq or varags for multiple executorIds,
you could push the these other updates down into it.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
