[
https://issues.apache.org/jira/browse/YARN-9161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16751934#comment-16751934
]
Zac Zhou commented on YARN-9161:
--------------------------------
[~sunilg], Thanks a lot for your comments~
{quote}I have some doubts here. Do we need this filtering?
if we use correct resource name in resource-types.xml for gpu and fpgs, below
code (in CapacitySchedulerConfigureation) could pick it up
{quote}
I think the filter is needed. The parameter, resourceTypes, of
updateResourceValuesFromConfig was based on AbsoluteResourceType. Now it's from
the following code in the method updateConfigurableResourceRequirement of
AbstractCSQueue
{code:java}
Set<String> resources = Arrays.stream(clusterResource.getResources()).map(x
-> x.getName()).collect(Collectors.toSet());
{code}
With the variable resources, we can get the resources in resource-types.xml.
If an incorrect resource is specified in capacity-scheduler.xml,
CapacitySchedulerConfigureation.updateResourceValuesFromConfig would ignore it.
So that the scheduler would not use the incorrect resource for container
allocation.
Hope I make it clear~
> Absolute resources of capacity scheduler doesn't support GPU and FPGA
> ---------------------------------------------------------------------
>
> Key: YARN-9161
> URL: https://issues.apache.org/jira/browse/YARN-9161
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
> Reporter: Zac Zhou
> Assignee: Zac Zhou
> Priority: Major
> Attachments: YARN-9161.001.patch, YARN-9161.002.patch,
> YARN-9161.003.patch, YARN-9161.004.patch, YARN-9161.005.patch,
> YARN-9161.006.patch
>
>
> As the enum CapacitySchedulerConfiguration.AbsoluteResourceType only has two
> elements: memory and vcores, which would filter out absolute resources
> configuration of gpu and fpga in
> AbstractCSQueue.updateConfigurableResourceRequirement.
> This issue would cause gpu and fpga can't be allocated correctly
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
