[
https://issues.apache.org/jira/browse/YARN-3432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15066306#comment-15066306
]
Akira AJISAKA commented on YARN-3432:
-------------------------------------
Thanks [~brahmareddy] for the comment and updating the patch.
bq. I feel bettter to include in TotalMB/Containers as reservedMB/Container
Do you mean you want to add "total containers" metric in ClusterMetrics? It
should be discussed in a separate jira.
I'm thinking "Total MB = Available MB + Allocated MB + Reserved MB" seems fine,
so I'd like to change how to calculate the metrics in FairScheduler. However,
it is an incompatible change.
Therefore, I'm okay with your (compatible) approach, but you need to fix the
following:
{code}
+ if (rs instanceof CapacityScheduler) {
+ this.totalMB = availableMB + allocatedMB + reservedMB;
+ this.totalVirtualCores = availableVirtualCores + allocatedVirtualCores
+ + containersReserved;
{code}
{{containersReserved}} should be {{reservedVirtualCores}}.
In addition, I'm thinking it's better to add a regression test to verify that
totalMB and totalVirtualCores are unchanged when a resource is reserved in both
CapacityScheduler and FairScheduler.
> Cluster metrics have wrong Total Memory when there is reserved memory on CS
> ---------------------------------------------------------------------------
>
> Key: YARN-3432
> URL: https://issues.apache.org/jira/browse/YARN-3432
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacityscheduler, resourcemanager
> Affects Versions: 2.6.0
> Reporter: Thomas Graves
> Assignee: Brahma Reddy Battula
> Attachments: YARN-3432-002.patch, YARN-3432-003.patch, YARN-3432.patch
>
>
> I noticed that when reservations happen when using the Capacity Scheduler,
> the UI and web services report the wrong total memory.
> For example. I have a 300GB of total memory in my cluster. I allocate 50
> and I reserve 10. The cluster metrics for total memory get reported as 290GB.
> This was broken by https://issues.apache.org/jira/browse/YARN-656 so perhaps
> there is a difference between fair scheduler and capacity scheduler.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
