Bobby Wang created SPARK-45495:
----------------------------------
Summary: Support stage level task resource profile for k8s cluster
when dynamic allocation disabled
Key: SPARK-45495
URL: https://issues.apache.org/jira/browse/SPARK-45495
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 3.4.1
Reporter: Bobby Wang
Assignee: Bobby Wang
Fix For: 4.0.0, 3.5.1
[https://github.com/apache/spark/pull/37268] has introduced a new feature that
supports stage-level schedule task resource profile for standalone cluster when
dynamic allocation is disabled. It's really cool feature, especially for ML/DL
cases, more details can be found in that PR.
The problem here is that the feature is only available for standalone cluster
for now, but most users would also expect it can be used for other spark
clusters like yarn and k8s.
So I file this issue to track this task.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
