Imran Rashid created SPARK-24016:
------------------------------------
Summary: Yarn does not update node blacklist in static allocation
Key: SPARK-24016
URL: https://issues.apache.org/jira/browse/SPARK-24016
Project: Spark
Issue Type: Improvement
Components: Scheduler, YARN
Affects Versions: 2.3.0
Reporter: Imran Rashid
Task-based blacklisting keeps track of bad nodes, and updates YARN with that
set of nodes so that Spark will not receive more containers on that node.
However, that only happens with dynamic allocation. Though its far more
important with dynamic allocation, even with static allocation this matters; if
executors die, or if the cluster was too busy at the original resource request
to give all the containers, the spark application will add new containers in
the middle. And we want an updated node blacklist for that.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
