[
https://issues.apache.org/jira/browse/YARN-8059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16658447#comment-16658447
]
Wilfred Spiegelenburg commented on YARN-8059:
---------------------------------------------
For the tests: I was expecting that when you use custom resources the custom
resource is dominant and or fully utilised etc. Otherwise it does not make
sense to test with custom resources configured. You changed the
{{testGetMaxUtilizationOfAllocatedResourcesWithAllocatedResource}} test but we
run other tests also specifically with custom resources.
The other tests with custom resources are most likely still using all memory
also and never check the custom resource changes.
The {{takeAllResourcesOfCustomResource}} calculates the containers based on
memory. You then calculate the amount of custom resources to allocate based on
the number of containers that fully allocates all memory. That is the part that
does not make sense to me. Why use memory as a base? I would expect that you
calculate the container size on the custom resource you want to use and make
sure that memory and cores are not a factor at all.
> Resource type is ignored when FS decide to preempt
> --------------------------------------------------
>
> Key: YARN-8059
> URL: https://issues.apache.org/jira/browse/YARN-8059
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
> Affects Versions: 3.0.0
> Reporter: Yufei Gu
> Assignee: Szilard Nemeth
> Priority: Major
> Attachments: YARN-8059.001.patch, YARN-8059.002.patch,
> YARN-8059.003.patch, YARN-8059.004.patch, YARN-8059.005.patch
>
>
> Method Fairscheduler#shouldAttemptPreemption doesn't consider resources other
> than vcore and memory. We may need to rethink it in the resource type
> scenario. cc [~miklos.szeg...@cloudera.com], [~wilfreds] and [~snemeth].
> {code}
> if (context.isPreemptionEnabled()) {
> return (context.getPreemptionUtilizationThreshold() < Math.max(
> (float) rootMetrics.getAllocatedMB() /
> getClusterResource().getMemorySize(),
> (float) rootMetrics.getAllocatedVirtualCores() /
> getClusterResource().getVirtualCores()));
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
