abhishek kumar tiwari created SPARK-35626:
---------------------------------------------
Summary: Decommission executors in batches to avoid overloading
network by block migrations.
Key: SPARK-35626
URL: https://issues.apache.org/jira/browse/SPARK-35626
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 3.1.2
Reporter: abhishek kumar tiwari
Currenlty, each executor is asked to starts offloading rdd and shuffle blocks
as soon it is decommissioned. This can overload the network bandwidth of the
application.
We should limit the number of executors migrating rdd and shuffle blocks at one
time. This would be configurable to enable users to decommission executors
conservatively or aggressively based on the use cases.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
