Just an update on this: turns out that in 2.2.0 version, the container
count is implemented by using the memory
resource, even though there are configs for vcores. Things might have
changed since
then.
Thanks for the suggestions. yarn.xml was a typo, should have been
yarn-site.xml.
About the heap
You also might want to increase values for mapreduce.{map,reduce}.memory.mb
to 1280 or 1536 or so (assuming that mapreduce.{map,reduce}.java.opts =
-Xmx1024m).
mapreduce.{map,reduce}.memory.mb is logical size of the container and it
should be larger than mapreduce.{map,reduce}.java.opts that speci
It should be yarn-site.xml not yarn.xml.
yarn.xml will not be added to $CLASSPATH
Thanks,
Wangda
On Wed, May 28, 2014 at 8:56 AM, hari wrote:
> The issue was not related the configuration related to containers. Due to
> misconfiguration, the Application master was not able to contact
> resource
Thanks. That might be reason why there are 64 containers instead of 16.
But, cgroups is not mentioned in a more recent article when discussing how
containers count is determined:
http://blog.cloudera.com/blog/2014/04/apache-hadoop-yarn-avoiding-6-time-consuming-gotchas/
.
It is also not mentioned
I believe it's behaving as expected. It will spawn 64 containers because
that's how much memory you have available. The vcores isn't harshly
enforced since CPUs can be elastic. This blog from cloudera explain how to
enforce CPU limits using CGroups.
http://blog.cloudera.com/blog/2013/12/managing-mu
The issue was not related the configuration related to containers. Due to
misconfiguration, the Application master was not able to contact
resourcemanager
causing in the 1 container problem.
However, the total containers allocated still is not as expected. The
configuration settings
should have re
Hi,
When using YARN 2.2.0 version, only 1 container is created
for an application in the entire cluster.
The single container is created at an arbitrary node
for every run. This happens when running any application from
the examples jar (e.g., wordcount). Currently only one application is
run at a
