I have configured Hadoop Yarn in a VM with 2048MB of RAM and 1 CPU core.
Then, I have configured the max and the min limits of memory to be used
in Yarn in the |mapred-site.xml|
|<property> <name>yarn.scheduler.minimum-allocation-mb</name>
<value>1024</value> </property> <property>
<name>yarn.scheduler.maximum-allocation-mb</name> <value>2048</value>
</property> <property>
<name>yarn.scheduler.minimum-allocation-vcores</name> <value>1</value>
</property> <property>
<name>yarn.scheduler.maximum-allocation-vcores</name> <value>1</value>
</property> <property> <name>yarn.nodemanager.resource.memory-mb</name>
<value>2048</value> </property> <property>
<name>yarn.nodemanager.resource.cpu-vcores</name> <value>1</value>
</property> |
But when I run an example, the count words on a file with 32K, the job
simply won’t run. It always stays at |0%|.
I have tried to decrease both values in the |mapred-site.xml| to |1024|
and |512|, respectively, but the job couldn’t also run. Therefore, I
have kept the |2048| and |1024| values.
To try to understand what is going on, I have looked to the Yarn logs,
but I haven’t find much useful information. I have just found was this
line, but it seems that everything is ok.
|2016-04-01 12:15:51,728 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl:
Starting resource-monitoring for container_1459527184896_0001_01_000001
2016-04-01 12:15:51,797 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl:
Memory usage of ProcessTree 29808 for container-id
container_1459527184896_0001_01_000001: 48.9 MB of 2 GB physical memory
used; 1.1 GB of 4.2 GB virtual memory used |
I don’t know why my example won’t work. Any help to debug this problem?
Is it possible to run a job in a VM with 2048GB of RAM and 1 CPU core?
