Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/7786
@vazin, sorry for the delay.
I think your concern is valid that this change will probably harm the
performance since 15 seconds (preemption waiting) is not a short time.
But your suggestion of killing the executors ahead of preemption to release
the resource is also risky, in Yarn's side preemption is based on the decision
at that time, the decision will be changed time to time since the resource is
dynamically changed, if later on the resource is enough that don't need to do
preemption, this pre-killing and re-acquire of containers will also harm the
performance to some extent.
Maybe current implementation is sufficient enough to address the preemption
problem, we don't need to do anything about it. I'm going to close this, thanks
a lot for your time and review.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
