Alex Milowski created SPARK-24491:
-------------------------------------
Summary: Support for requesting GPU resources on K8S
Key: SPARK-24491
URL: https://issues.apache.org/jira/browse/SPARK-24491
Project: Spark
Issue Type: New Feature
Components: Kubernetes
Affects Versions: 2.3.0
Reporter: Alex Milowski
If GPU resources are required for executor PODS in a multi-tentan k8s cluster,
the resource limits section needs to specify that GPUs are required [1]. This
is a simple request:
{{resources:}}
{{ limits:}}
{{ nvidia.com/gpu: 1 # requesting 1 GPU}}
A simple configuration limit could be specified (just like executor memory):
{{ --conf spark.kubernetes.executor.limit.gpus=1}}
[1] [https://kubernetes.io/docs/tasks/manage-gpus/scheduling-gpus/]
I have an implementation I will submit via a pull request.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
