[
https://issues.apache.org/jira/browse/SPARK-38047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Holmes updated SPARK-38047:
--------------------------------
Description:
Currently executor rolling will always kill one executor every
{{{}spark.kubernetes.executor.rollInterval{}}}. For some of the policies this
may not be optimal in cases where the executor metric isn't an outlier compared
to other executors. There is a cost associated with killing executors (ramp-up
time for new executors for example) which applications may not want to incur
for non-outlier executors.
This ticket would add the ability to only kill executors if they are outliners.
was:
Currently executor rolling will always kill one executor every
{{{}spark.kubernetes.executor.rollInterval{}}}. For some of the policies this
may not be optimal in cases where the executor metric isn't an outlier compared
to other executors.
This ticket would add the ability to only kill executors if they are outliners.
> Provide an option to only roll executors if they are outliers
> -------------------------------------------------------------
>
> Key: SPARK-38047
> URL: https://issues.apache.org/jira/browse/SPARK-38047
> Project: Spark
> Issue Type: Sub-task
> Components: Kubernetes
> Affects Versions: 3.3.0
> Reporter: Alex Holmes
> Priority: Major
> Fix For: 3.3.0
>
>
> Currently executor rolling will always kill one executor every
> {{{}spark.kubernetes.executor.rollInterval{}}}. For some of the policies this
> may not be optimal in cases where the executor metric isn't an outlier
> compared to other executors. There is a cost associated with killing
> executors (ramp-up time for new executors for example) which applications may
> not want to incur for non-outlier executors.
>
> This ticket would add the ability to only kill executors if they are
> outliners.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
