Repository: spark Updated Branches: refs/heads/master dc6c5ea4c -> 7a6ee8a8f
[SPARK-12009][YARN] Avoid to re-allocating yarn container while driver want to stop all Executors Author: hushan <hus...@xiaomi.com> Closes #9992 from suyanNone/tricky. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/7a6ee8a8 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/7a6ee8a8 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/7a6ee8a8 Branch: refs/heads/master Commit: 7a6ee8a8fe0fad78416ed7e1ac694959de5c5314 Parents: dc6c5ea Author: hushan <hus...@xiaomi.com> Authored: Thu Feb 25 16:57:41 2016 -0800 Committer: Andrew Or <and...@databricks.com> Committed: Thu Feb 25 16:57:41 2016 -0800 ---------------------------------------------------------------------- .../org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala | 3 +++ 1 file changed, 3 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/7a6ee8a8/yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala ---------------------------------------------------------------------- diff --git a/yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala b/yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala index 1431bce..ca26277 100644 --- a/yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala +++ b/yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala @@ -83,6 +83,9 @@ private[spark] abstract class YarnSchedulerBackend( override def stop(): Unit = { try { + // SPARK-12009: To prevent Yarn allocator from requesting backup for the executors which + // was Stopped by SchedulerBackend. + requestTotalExecutors(0, 0, Map.empty) super.stop() } finally { services.stop() --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org