Thanks for the reply. So what is the purpose of heap sizes for tasktrackers and datanodes then? In other words, if I want to speed up the map/reducing cycle, can I just minimize the heap size and maximize the "mapred.child.java.opts?" or will the minimizing heap sizes causing out of memory exception?
On Mon, Nov 25, 2013 at 10:02 AM, Kai Voigt <k...@123.org> wrote: > mapred.child.java.opts are referring to the settings for the JVMs spawned > by the TaskTracker. This JVMs will actually run the tasks (mappers and > reducers) > > The heap sizes for TaskTrackers and DataNodes are unrelated to those. They > run in their own JVMs each. > > Kai > > Am 25.11.2013 um 15:52 schrieb Chih-Hsien Wu <chjaso...@gmail.com>: > > I'm learning about Hadoop configuration. What is the connection between > the datanode/ tasktracker heap sizes and the "mapre.child.java.opts"? Does > one have to be exceeded to another? > > > ------------------------------ > *Kai Voigt* Am Germaniahafen 1 k...@123.org > 24143 Kiel +49 160 96683050 > Germany @KaiVoigt > >
