Andrew Or created SPARK-5698:
--------------------------------
Summary: Dynamic allocation: do not allow user to request a
negative delta
Key: SPARK-5698
URL: https://issues.apache.org/jira/browse/SPARK-5698
Project: Spark
Issue Type: Bug
Components: Spark Core, YARN
Affects Versions: 1.2.0
Reporter: Andrew Or
Assignee: Andrew Or
Priority: Critical
If the user calls the following in series:
sc.requestExecutors(-5)
sc.killExecutor(...)
Then we might crash the ApplicationMaster. Why?
Well, if we request a negative number of additional executors, then the
YarnAllocator will lower the target number of executors it is trying to achieve
by 5. This might shoot the target number past 0. Then, if so, when we try to
kill an executor it will fail the assertion that the target number must be >=0.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
