tgravescs commented on a change in pull request #25047: [WIP][SPARK-27371][CORE] Support GPU-aware resources scheduling in Standalone URL: https://github.com/apache/spark/pull/25047#discussion_r305870426
########## File path: core/src/main/scala/org/apache/spark/resource/ResourceUtils.scala ########## @@ -70,6 +93,276 @@ private[spark] object ResourceUtils extends Logging { // internally we currently only support addresses, so its just an integer count val AMOUNT = "amount" + /** + * Assign resources to workers/drivers from the same host to avoid address conflict. + * + * This function works in three steps. First, acquiring the lock on RESOURCES_LOCK_FILE + * to achieve synchronization among workers and drivers. Second, getting all allocated + * resources from ALLOCATED_RESOURCES_FILE and assigning isolated resources to the worker + * or driver after differentiating available resources in discovered resources from + * allocated resources. If available resources don't meet worker's or driver's requirement, + * try to update allocated resources by excluding the resource allocation if its related + * process has already terminated and do the assignment again. If still don't meet requirement, + * exception would be threw. Third, updating ALLOCATED_RESOURCES_FILE with new allocated Review comment: nit: s/threw/thrown ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org