[ https://issues.apache.org/jira/browse/YARN-5551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437415#comment-15437415 ]
Nathan Roberts commented on YARN-5551: -------------------------------------- I think the two examples you provided in the description are actually 2 very different cases. Notice how the first has an anonymous size of 0 while the second has the entire dirty region marked as anonymous. I think (not certain here), that this means in the first case the kernel actually has file-backed pages to write to if necessary. In the second case, I feel like anonymous means it does NOT have a place to put dirty pages (like maybe the file has been both truncated and unlinked). If that's a correct interpretation of "anonymous" then I feel like we should be counting the second mapping in the processes memory usage. > Ignore deleted file mapping from memory computation when smaps is enabled > ------------------------------------------------------------------------- > > Key: YARN-5551 > URL: https://issues.apache.org/jira/browse/YARN-5551 > Project: Hadoop YARN > Issue Type: Improvement > Reporter: Rajesh Balamohan > Assignee: Rajesh Balamohan > Priority: Minor > Attachments: YARN-5551.branch-2.001.patch > > > Currently deleted file mappings are also included in the memory computation > when SMAP is enabled. For e.g > {noformat} > 7f612004a000-7f612004c000 rw-s 00000000 00:10 4201507513 > /dev/shm/HadoopShortCircuitShm_DFSClient_NONMAPREDUCE_-521969216_162_734673185 > (deleted) > Size: 8 kB > Rss: 4 kB > Pss: 2 kB > Shared_Clean: 0 kB > Shared_Dirty: 4 kB > Private_Clean: 0 kB > Private_Dirty: 0 kB > Referenced: 4 kB > Anonymous: 0 kB > AnonHugePages: 0 kB > Swap: 0 kB > KernelPageSize: 4 kB > MMUPageSize: 4 kB > 7f6123f99000-7f6163f99000 rw-p 00000000 08:41 211419477 > /grid/4/hadoop/yarn/local/usercache/root/appcache/application_1466700718395_1249/container_e19_1466700718395_1249_01_000003/7389389356021597290.cache > (deleted) > Size: 1048576 kB > Rss: 637292 kB > Pss: 637292 kB > Shared_Clean: 0 kB > Shared_Dirty: 0 kB > Private_Clean: 0 kB > Private_Dirty: 637292 kB > Referenced: 637292 kB > Anonymous: 637292 kB > AnonHugePages: 0 kB > Swap: 0 kB > KernelPageSize: 4 kB > {noformat} > It would be good to exclude these from getSmapBasedRssMemorySize() > computation. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org