On 3/30/10 10:15 AM, "Vasilis Liaskovitis" <vlias...@gmail.com> wrote:
> - Focusing on a single job config, Is there a rule of thumb about how
> much node memory should be left for use outside of Child JVMs?
> I make sure that per Node, there is free memory:
> (#maxmapTasksperTaskTracker + #maxreduceTasksperTaskTracker) *
> JVMHeapSize < PhysicalMemoryonNode
> The total JVM heap size per node per job from the above equation
> currently account 65%-75% of the node's memory. (I 've tried
> allocating a riskier 90% of the node's memory, with similar swapping
> observations).
Java takes more RAM than just the heap size. Sometimes 2-3x as much.
My general rule of thumb for general purpose grids is to plan on having
3-4gb of free VM (swap+physical) space for the OS, monitoring, datanode, and
task tracker processes. After that, you can carve it up however you want.
If you are running on machines with less than that much, you likely are
going to face at least a little bit of swapping or have minimum monitoring
or whatever.
