ivoson commented on code in PR #36716:
URL: https://github.com/apache/spark/pull/36716#discussion_r893565176
##########
core/src/main/scala/org/apache/spark/deploy/master/Master.scala:
##########
@@ -725,26 +729,38 @@ private[deploy] class Master(
*/
private def startExecutorsOnWorkers(): Unit = {
// Right now this is a very simple FIFO scheduler. We keep trying to fit
in the first app
- // in the queue, then the second app, etc.
+ // in the queue, then the second app, etc. And for each app, we will
schedule base on
+ // resource profiles also with a simple FIFO scheduler, resource profile
with smaller id
+ // first.
Review Comment:
Yes, you are right. Smaller id does not indicate higher priority directly.
The strategy is simple and straightforward, relying on the dynamic allocation
manager to adjust the target number for each resource profile, and we just
fulfill the requirements in a fixed order.
Currently, we don't have a good way to infer about the order of requests for
different resource profiles. The information we have are the target number and
existing executor number for each resource profile. Maybe we can compute the
priority somehow in a FAIR schedule mode.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
