For us we typically leave a 500MB difference between the heap and the container size. I think we can make this smaller, but we have not really tried.
--Bobby On 5/3/13 11:20 AM, "Karthik Kambatla" <ka...@cloudera.com> wrote: >Hi > >While looking into MAPREDUCE-5207 (adding defaults for >mapreduce.{map|reduce}.memory.mb), I was wondering how much headroom >should >be left on top of mapred.child.java.opts (or other similar JVM opts) for >the container memory itself? > >Currently, mapred.child.java.opts (per mapred-default.xml) is set to 200 >MB >by default. The default for mapreduce.{map|reduce}.memory.mb is 1024 in >the >code, which is significantly higher than the 200MB value. > >Do we need more than 100 MB for non-JVM memory per container? If so, does >it make sense make that a config property in itself and the code to verify >all 3 values are clear enough? > >Thanks >Karthik