[
https://issues.apache.org/jira/browse/SPARK-21833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16140533#comment-16140533
]
Sital Kedia commented on SPARK-21833:
-------------------------------------
Actually, SPARK-20540 already addressed this issue on latest trunk. Will close
this issue.
> CoarseGrainedSchedulerBackend leaks executors in case of dynamic allocation
> ---------------------------------------------------------------------------
>
> Key: SPARK-21833
> URL: https://issues.apache.org/jira/browse/SPARK-21833
> Project: Spark
> Issue Type: Bug
> Components: Scheduler
> Affects Versions: 2.2.0
> Reporter: Sital Kedia
>
> We have seen this issue in coarse grained scheduler that in case of dynamic
> executor allocation is turned on, the scheduler asks for more executors than
> needed. Consider the following situation where there are excutor allocation
> manager is ramping down the number of executors. It will lower the executor
> targer number by calling requestTotalExecutors api.
> Later, when the allocation manager finds some executors to be idle, it will
> call killExecutor api. The coarse grain scheduler, in the killExecutor
> function replaces the total executor needed to current + pending which
> overrides the earlier target set by the allocation manager.
> This results in scheduler spawning more executors than actually needed.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
