[ https://issues.apache.org/jira/browse/TEZ-3647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15895076#comment-15895076 ]
Siddharth Seth commented on TEZ-3647: ------------------------------------- +1. Thanks [~aplusplus]. Committing. > Lower default Xmx for small AM container > ----------------------------------------- > > Key: TEZ-3647 > URL: https://issues.apache.org/jira/browse/TEZ-3647 > Project: Apache Tez > Issue Type: Bug > Reporter: Zhiyuan Yang > Assignee: Zhiyuan Yang > Attachments: TEZ-3647.1.patch, TEZ-3647.2.patch, TEZ-3647.3.patch > > > The default Xmx is 80% of container size. For a 2G container, non-heap memory > can cost more than 20%. Total off-heap memory is around 450MB. In case of > nearly full heap memory usage, we will be out of physical memory and killed > by YARN. > Here is native memory summary for a AM running tpcds query 13. > {code} > Native Memory Tracking: > Total: reserved=3424708KB, committed=2155712KB > - Java Heap (reserved=1677312KB, committed=1677312KB) > (mmap: reserved=1677312KB, committed=1677312KB) > - Class (reserved=1096701KB, committed=54061KB) > (classes #8465) > (malloc=1021KB #11433) > (mmap: reserved=1095680KB, committed=53040KB) > - Thread (reserved=195103KB, committed=195103KB) > (thread #191) > (stack: reserved=193948KB, committed=193948KB) > (malloc=611KB #986) > (arena=543KB #382) > - Code (reserved=254302KB, committed=31222KB) > (malloc=4702KB #7076) > (mmap: reserved=249600KB, committed=26520KB) > - GC (reserved=148963KB, committed=148963KB) > (malloc=53951KB #17562) > (mmap: reserved=95012KB, committed=95012KB) > - Compiler (reserved=306KB, committed=306KB) > (malloc=176KB #311) > (arena=131KB #3) > - Internal (reserved=17872KB, committed=17872KB) > (malloc=17840KB #15275) > (mmap: reserved=32KB, committed=32KB) > - Symbol (reserved=12659KB, committed=12659KB) > (malloc=10829KB #106221) > (arena=1830KB #1) > - Native Memory Tracking (reserved=2735KB, committed=2735KB) > (malloc=192KB #2968) > (tracking overhead=2543KB) > - Arena Chunk (reserved=5034KB, committed=5034KB) > (malloc=5034KB) > - Tracing (reserved=10444KB, committed=10444KB) > (malloc=10444KB #274) > - Unknown (reserved=3276KB, committed=0KB) > (mmap: reserved=3276KB, committed=0KB) > {code} > Please notice the memory here means virtual memory size. In worst scenario > all pages get loaded and takes same amount of physical memory. -- This message was sent by Atlassian JIRA (v6.3.15#6346)