[
https://issues.apache.org/jira/browse/SPARK-28022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Henry Yu updated SPARK-28022:
-----------------------------
Description:
Hi, in order to achieve cloud native friendly autoscaling , I propose to add a
pod affinity feature.
Traditionally, when we use spark in fix size yarn cluster, it make sense to
spread containers to every node.
Coming to cloud native resource manage, we want to release node when we don't
need it any more.
Pod affinity feature counts to place all pods of certain application to some
nodes instead of all nodes.
By the way, using pod template is not a good choice, for add application id
to pod affinity term when submit is more robust.
was:
Hi, in order to achieve cloud native friendly autoscaling , I propose to add a
pod affinity feature.
Traditionally, when we use spark in fix size yarn cluster, it make sense to
spread containers to every node.
Coming to cloud native resource manage, we want to release node when we don't
need it any more.
Pod affinity feature counts to place all pods of certain application to some
nodes instead of all nodes.
By the way, we can not use pod template to achieve this, for we must add
application id to affinity term .
> k8s pod affinity achieve cloud native friendly autoscaling
> -----------------------------------------------------------
>
> Key: SPARK-28022
> URL: https://issues.apache.org/jira/browse/SPARK-28022
> Project: Spark
> Issue Type: New Feature
> Components: Kubernetes
> Affects Versions: 2.4.3
> Reporter: Henry Yu
> Priority: Major
>
> Hi, in order to achieve cloud native friendly autoscaling , I propose to add
> a pod affinity feature.
> Traditionally, when we use spark in fix size yarn cluster, it make sense to
> spread containers to every node.
> Coming to cloud native resource manage, we want to release node when we don't
> need it any more.
> Pod affinity feature counts to place all pods of certain application to some
> nodes instead of all nodes.
> By the way, using pod template is not a good choice, for add application id
> to pod affinity term when submit is more robust.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
