tgravescs commented on a change in pull request #26078: [SPARK-29151][CORE]
Support fractional resources for task resource scheduling
URL: https://github.com/apache/spark/pull/26078#discussion_r333564756
##########
File path: docs/configuration.md
##########
@@ -1982,9 +1982,13 @@ Apart from these, the following properties are also
available, and may be useful
<td><code>spark.task.resource.{resourceName}.amount</code></td>
<td>1</td>
<td>
- Amount of a particular resource type to allocate for each task. If this is
specified
- you must also provide the executor config
<code>spark.executor.resource.{resourceName}.amount</code>
- and any corresponding discovery configs so that your executors are created
with that resource type.
+ Amount of a particular resource type to allocate for each task, note that
this can be a double.
+ If this is specified you must also provide the executor config
+ <code>spark.executor.resource.{resourceName}.amount</code> and any
corresponding discovery configs
+ so that your executors are created with that resource type. In addition to
whole amounts,
+ a fractional amount (for example, 0.25, or 1/4th of a resource) may be
specified.
+ Fractional amounts must be less than or equal to 0.5, or in other words,
the minimum amount of
Review comment:
we should also add that any fractional value specified gets rounded to have
an even number of tasks per resource. ie 0.2222 end up with 4 tasks per
resource
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
