Github user smurakozi commented on a diff in the pull request:
https://github.com/apache/spark/pull/20203#discussion_r161005667
--- Diff:
core/src/main/scala/org/apache/spark/status/AppStatusListener.scala ---
@@ -223,6 +228,15 @@ private[spark] class AppStatusListener(
updateNodeBlackList(event.hostId, false)
}
+ def updateBlackListStatusForStage(executorId: String, stageId: Int,
stageAttemptId: Int): Unit = {
+ Option(liveStages.get((stageId, stageAttemptId))).foreach { stage =>
+ val now = System.nanoTime()
+ val esummary = stage.executorSummary(executorId)
+ esummary.isBlacklisted = true
+ maybeUpdate(esummary, now)
+ }
+ }
+
--- End diff --
LiveEntities periodically write an immutable view of the entity to the
store. LiveExecutor and LiveExecutorStageSummary were modified in this PR to
maintain blacklisted status and to write it to the state store.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
