wbo4958 commented on code in PR #45631:
URL: https://github.com/apache/spark/pull/45631#discussion_r1535100931
##########
core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala:
##########
@@ -2654,18 +2659,21 @@ class TaskSchedulerImplSuite extends SparkFunSuite with
LocalSparkContext
taskScheduler.submitTasks(lowerTaskSet)
taskScheduler.submitTasks(higherRpTaskSet)
- // only offer the resources to the higher taskset
+ // Initially, offer the available resources to the higher priority task
set, where each task
+ // requires 0.7 GPU, so Spark can assign the GPU resources to a
maximum of 4 tasks, leaving
+ // 0.3 GPU available for each remaining tasks.
+ // Secondly, try to offer the available resources to the lower priority
task set, where each
+ // task requires 0.4 GPU, while only 0.3 gpu of each address is
available, so couldn't
+ // offer any resources for the lower tasks.
Review Comment:
LowerTaskSet is indeed submitted before higherRpTaskSet, but there's no
resource to offer for lowerTaskSet at that time, lowerTaskSet will be put into
the Pool (using FIFO algo by default). After both lowerTaskSet and
higherRpTaskSet are in the Pool, the test case begins to offer resources and
spark will choose the higher TaskSet (in this case, it is higherTaskSet) to
offer. So it should behave like the comments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
