[
https://issues.apache.org/jira/browse/SPARK-38397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-38397:
----------------------------------
Description:
There are several ways to run Spark on K8s including vanilla `spark-submit`
with built-in `KubernetesClusterManager`, `spark-submit` with custom
`ExternalClusterManager`, CRD-based operators (like spark-on-k8s-operator),
custom K8s `schedulers`, custom `standalone pod definitions`, and so on.
This issue is tracking K8s-native Job Queueing related work.
* [https://github.com/kubernetes-sigs/kueue]
{code}
metadata:
generateName: sample-job-
annotations:
kueue.k8s.io/queue-name: main
{code}
The best case is Apache Spark users use it in the future via pod templates or
existing configuration. In other words, we don't need to do anything and close
this JIRA without any patches.
was:
This issue is tracking K8s-native Job Queueing related work.
* [https://github.com/kubernetes-sigs/kueue]
{code}
metadata:
generateName: sample-job-
annotations:
kueue.k8s.io/queue-name: main
{code}
The best case is Apache Spark users use it in the future via pod templates or
existing configuration. In other words, we don't need to do anything and close
this JIRA without any patches.
> Support Kueue: K8s-native Job Queueing
> --------------------------------------
>
> Key: SPARK-38397
> URL: https://issues.apache.org/jira/browse/SPARK-38397
> Project: Spark
> Issue Type: Improvement
> Components: Kubernetes
> Affects Versions: 3.4.0
> Reporter: Dongjoon Hyun
> Priority: Major
>
> There are several ways to run Spark on K8s including vanilla `spark-submit`
> with built-in `KubernetesClusterManager`, `spark-submit` with custom
> `ExternalClusterManager`, CRD-based operators (like spark-on-k8s-operator),
> custom K8s `schedulers`, custom `standalone pod definitions`, and so on.
> This issue is tracking K8s-native Job Queueing related work.
> * [https://github.com/kubernetes-sigs/kueue]
> {code}
> metadata:
> generateName: sample-job-
> annotations:
> kueue.k8s.io/queue-name: main
> {code}
> The best case is Apache Spark users use it in the future via pod templates or
> existing configuration. In other words, we don't need to do anything and
> close this JIRA without any patches.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
