[ https://issues.apache.org/jira/browse/YARN-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15592555#comment-15592555 ]
Kuhu Shukla commented on YARN-5356: ----------------------------------- I think that if we do not have the values from the plugin we could initialize the physicalResource to totalNodeResource (from conf values) instead of zero. May be something like : {code} int physicalMemoryMb; int physicalCores; if (rcp != null) { physicalMemoryMb = (int) rcp.getPhysicalMemorySize() / (1024 * 1024); physicalCores = rcp.getNumProcessors(); } else { physicalMemoryMb = conf.getInt( YarnConfiguration.NM_PMEM_MB, YarnConfiguration.DEFAULT_NM_PMEM_MB) + conf.getInt( YarnConfiguration.NM_SYSTEM_RESERVED_PMEM_MB, 0); ...... } {code} > NodeManager should communicate physical resource capability to ResourceManager > ------------------------------------------------------------------------------ > > Key: YARN-5356 > URL: https://issues.apache.org/jira/browse/YARN-5356 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager, resourcemanager > Affects Versions: 3.0.0-alpha1 > Reporter: Nathan Roberts > Assignee: Inigo Goiri > Attachments: YARN-5356.000.patch, YARN-5356.001.patch, > YARN-5356.002.patch, YARN-5356.002.patch, YARN-5356.003.patch, > YARN-5356.004.patch, YARN-5356.005.patch > > > Currently ResourceUtilization contains absolute quantities of resource used > (e.g. 4096MB memory used). It would be good if the NM also communicated the > actual physical resource capabilities of the node so that the RM can use this > data to schedule more effectively (overcommit, etc) > Currently the only available information is the Resource the node registered > with (or later updated using updateNodeResource). However, these aren't > really sufficient to get a good view of how utilized a resource is. For > example, if a node reports 400% CPU utilization, does that mean it's > completely full, or barely utilized? Today there is no reliable way to figure > this out. > [~elgoiri] - Lots of good work is happening in YARN-2965 so curious if you > have thoughts/opinions on this? -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org