[ https://issues.apache.org/jira/browse/YARN-4768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nathan Roberts updated YARN-4768: --------------------------------- Attachment: YARN-4768.patch Patch for trunk. Also changed getPhysicalMemorySize() to exclude: - HardwareCorrupted pages - Not that uncommon. - HugePagesTotal * hugePageSize - probably not commonly configured on compute nodes but just in case it seems reasonable to not count these. Comments welcome on alternative ways to approach these. > getAvailablePhysicalMemorySize can be inaccurate on linux > --------------------------------------------------------- > > Key: YARN-4768 > URL: https://issues.apache.org/jira/browse/YARN-4768 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager > Affects Versions: 3.0.0, 2.7.2 > Environment: Linux > Reporter: Nathan Roberts > Assignee: Nathan Roberts > Attachments: YARN-4768.patch > > > Algorithm currently uses "MemFree" + "Inactive" from /proc/meminfo > "Inactive" may not be a very good indication of how much memory can be > readily freed because it contains both: > - Pages mapped with MAP_SHARED|MAP_ANONYMOUS (regardless of whether they're > being actively accessed or not. Unclear to me why this is the case...) > - Pages mapped MAP_PRIVATE|MAP_ANONYMOUS that have not been accessed recently > Both of these types of pages probably shouldn't be considered "Available". > "Inactive(file)" would seem more accurate but it's not available in all > kernel versions. To keep things simple, maybe just use "Inactive(file)" if > available, otherwise fallback to "Inactive". -- This message was sent by Atlassian JIRA (v6.3.4#6332)