Hi Arun
Thanks a lot.
With a container size of 2G, the max heap size i can allocate is ~1.6G.
is the heap exclusively possessed by the container ,or shared by all
containers
which belong to a AM.
I also notice that there are several containers allocated in a NM.
however only one or a few of them would process the job,
(State:NEW --> LOCALIZING --> LOCALIZED --> RUNNING --> KILLING -->
EXITED_WITH_SUCCESS) ,
Others were killed (State:NEW --> LOCALIZING --> LOCALIZED --> RUNNING
--> KILLING --> CONTAINER_CLEANEDUP_AFTER_KILL--> DONE).
So i wonder what is the container allocation policy,and how many
containers would be allocated when a job is submitted.
Regards
2013/2/4 Arun C Murthy <a...@hortonworks.com>
> Youpeng,
>
> The *container* is a generic concept which allows the ResourceManager to
> provide you a specific amount of resources (memory, cpu etc.).
>
> mapred.map.child.java.opts is part of the cmd-line of the Map task's jvm
> which specifies the heap size.
>
> So, in YARN, mapred.map.child.java.opts should be less than memory
> allocated to container.
>
> Remember that container memory is physical memory of the unix process,
> while mapred.map.child.java.opts is heap size which doesn't account for
> non-heap memory.
>
> So, with a container size of 2G, the max heap size you can allocate is
> ~1.6G or so.
>
> hth,
> Arun
>
> On Feb 4, 2013, at 3:38 AM, YouPeng Yang wrote:
>
> Hi All
>
> I use YARN and MR.I am confused by the relationship between
> mapred.map.child.java.opts and Container resource allocation.
> if I setup mapred.map.child.java.opts = -Xmx2048m ,on nodemanager
> how Containers will be allocated according to this settings.
> what about mapred.reduce.child.java.opts?
>
> regards
> Youpeng Yang
>
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
