Thomas Graves created SPARK-30749:
-------------------------------------
Summary: Better cleanup of Resource profiles
Key: SPARK-30749
URL: https://issues.apache.org/jira/browse/SPARK-30749
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 3.1.0
Reporter: Thomas Graves
In the initial stage level scheduling jiras we aren't doing a great job of
cleaning up datastructures when the ResourceProfile is done being used. This is
mostly because its hard to tell when its done being used.
We should find a way to clean up better.
Also in the dynamic allocation manager, if the resource profile is not being
used anymore we should be more active about getting rid of the executors that
are up. Especially if there is a minimum number that is set, we can kill those
off.
I think this can be done as followup to the main feature
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
