[
https://issues.apache.org/jira/browse/HADOOP-5262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12673558#action_12673558
]
Matei Zaharia commented on HADOOP-5262:
---------------------------------------
I suggested we allow both modes because we have been using the
fixed-number-of-slots model at Facebook. We mostly use min share to guarantee
that ETL jobs and other jobs with deadlines get enough slots to finish in time.
Thus it makes more sense to set the min share to something small but just
enough to be sure that the job will finish, and this is easier to think of in
terms of slots than percents. I understand that the Yahoo use case is different
and requires assigning percent shares to organizations.
> Allow specifying min shares as percentage of cluster
> ----------------------------------------------------
>
> Key: HADOOP-5262
> URL: https://issues.apache.org/jira/browse/HADOOP-5262
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/fair-share
> Reporter: Matei Zaharia
> Priority: Minor
>
> Currently the guaranteed shares for pools in the fair scheduler are specified
> as a number of slots. For organizations where a group pays X% of the cluster
> and the actual number of nodes in the cluster varies due to failures,
> expansion, etc over time, it would be useful to support a guaranteed share
> given as a percentage too. This would just let you write in the config file
> something like <minMaps>5%</minMaps> instead of <minMaps>42</minMaps>. The
> scheduler would need to recompute what this means in terms of number of slots
> on every update (probably through some kind of update(ClusterStatus) method
> in PoolManager).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
