[ https://issues.apache.org/jira/browse/SPARK-30271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
angerszhu resolved SPARK-30271. ------------------------------- Resolution: Fixed > dynamic allocation won't release some executor in some case. > ------------------------------------------------------------ > > Key: SPARK-30271 > URL: https://issues.apache.org/jira/browse/SPARK-30271 > Project: Spark > Issue Type: Bug > Components: Scheduler, Spark Core > Affects Versions: 2.4.0 > Reporter: angerszhu > Priority: Major > > Case : > max executor 5 > min executor 0 > idle time 5s > stage-1 10 tasks run in 5 executors. > If stage-1 finished in 5 all executors, all executor added to `removeTimes` > when taskEnd event. > After 5s, start release process, since stage-2 have 20 tasks, then executor > won't be removed since existing executor num < executorTargetNum., and > executor will be removed from `removeTimes`. > But if task won't be scheduled to all these executors, if executor-1 won't > have task to run in it, it won't be put into `removeTimes` and if there are > no more tasks, executor won't be removed forever -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org