[
https://issues.apache.org/jira/browse/YARN-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13539427#comment-13539427
]
Karthik Kambatla commented on YARN-2:
-------------------------------------
Thanks Arun. The patch looks great, quite excited to see multi-resource
scheduling.
Few comments (mostly nits):
# Should we set the total # of virtual cores to 1024 (parity with cgroups), and
hence, set the virtual-to-physical ratio statically at startup. For instance,
should we operate on a machine with 8 cores, we set the ratio to 128.
# ResourceCalculator#divideAndCeil: LOG.debug() seems more appropriate than
LOG.info()
# Should we rename getResourceAsValue() to getResourceAsNormalizedValue() for
better readability?
# Following up on earlier discussions about this, passing {{int
dominanceLevel}} instead of {{boolean dominant}} address both performance and
API concerns. For now, we can check for (dominanceLevel == 1)
# MultiResourceCalculator does show different implementations of divide() and
ratio(), but it is hard to understand the difference from code or comments. May
be, we need better names to describe exactly what they do.
# Also, DefaultResourceCalculator defines ratio() in terms of divide().
Theoretically, shouldn't it be the other way round? I might be wrong here, but
I am still confused about the two.
# CSQueueUtils and LeafQueue seem to be using divide() and ratio()
interchangeably
# Should make the stepFactor argument names consistent across
ResourceCalculator, DefaultCalculator and MultiResourceCalculator. stepFactor
and factor might mean different things.
# FairScheduler comments - shouldn't the comment be saying "ensuring" instead
of "insuring"?
> Enhance CS to schedule accounting for both memory and cpu cores
> ---------------------------------------------------------------
>
> Key: YARN-2
> URL: https://issues.apache.org/jira/browse/YARN-2
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: capacityscheduler, scheduler
> Reporter: Arun C Murthy
> Assignee: Arun C Murthy
> Fix For: 2.0.3-alpha
>
> Attachments: MAPREDUCE-4327.patch, MAPREDUCE-4327.patch,
> MAPREDUCE-4327.patch, MAPREDUCE-4327-v2.patch, MAPREDUCE-4327-v3.patch,
> MAPREDUCE-4327-v4.patch, MAPREDUCE-4327-v5.patch, YARN-2-help.patch,
> YARN-2.patch, YARN-2.patch, YARN-2.patch, YARN-2.patch, YARN-2.patch
>
>
> With YARN being a general purpose system, it would be useful for several
> applications (MPI et al) to specify not just memory but also CPU (cores) for
> their resource requirements. Thus, it would be useful to the
> CapacityScheduler to account for both.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
