Hua Zhang created SPARK-27104:
---------------------------------
Summary: Spark Fair scheduler across applications in standalone
mode
Key: SPARK-27104
URL: https://issues.apache.org/jira/browse/SPARK-27104
Project: Spark
Issue Type: Wish
Components: Scheduler
Affects Versions: 2.4.0, 2.3.3, 2.2.3
Reporter: Hua Zhang
Spark in standalone mode currently only supports FIFO (first-in-first-out)
scheduler across applications.
It will be great that a fair scheduler is supported. +A fair scheduler across
applications, not in a application.+
Use case (for example with the integration of zeppelin)
At certain moment, user A submits an heavy application en consumes all the
resources of the spark cluster.
At a later moment, user B submits a second application.
No matter how many work nodes you added now, all the resources go to user A due
to the FIFO. User B will never get any resource until user A release its
allocated resources.
A fair scheduler should distribute extra resources in a fair way on all running
applications, which demands resources.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
