You need to change the application configuration itself to tell YARN that each task needs more than the default. I see that this is a mapreduce app, so you have to change the per-application configuration: mapreduce.map.memory.mb and mapreduce.reduce.memory.mb in either mapred-site.xml or via the command line.
Side notes: Seems like you are spawning lots of shells under your mapper and YARN's NodeManager is detecting that the total virtual memory usage is 14.5GB. You may want to reduce that number of shells, lest the OS itself might kill your tasks depend on the system configuration. Thanks, +Vinod On Jan 1, 2014, at 7:50 PM, S.L <simpleliving...@gmail.com> wrote: > Hello Folks, > > I am running hadoop 2.2 in a pseudo-distributed mode on a laptop with 8GB > RAM. > > Whenever I submit a job I get an error that says that the that the virtual > memory usage exceeded , like below. > > I have changed the ratio yarn.nodenamager.vmem-pmem-ratio in yarn-site.xml to > 10 , however the virtual memory is not getting increased more than 2.1 GB , > as can been seen in the error message below and the container is being killed. > > Can some one please let me know if there is any other setting that needs to > be changed ? Thanks in advance! > > Error Message : > > INFO mapreduce.Job: Task Id : attempt_1388632710048_0009_m_000000_2, Status : > FAILED > Container [pid=12013,containerID=container_1388632710048_0009_01_000004] is > running beyond virtual memory limits. Current usage: 544.9 MB of 1 GB > physical memory used; 14.5 GB of 2.1 GB virtual memory used. Killing > container. > Dump of the process-tree for container_1388632710048_0009_01_000004 : > |- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) > SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE > |- 12077 12018 12013 12013 (phantomjs) 16 2 1641000960 6728 > /usr/local/bin/phantomjs --webdriver=15358 > --webdriver-logfile=/tmp/hadoop-general/nm-local-dir/usercache/general/appcache/application_1388632710048_0009/container_1388632710048_0009_01_000004/phantomjsdriver.log > > |- 12013 882 12013 12013 (bash) 1 0 108650496 305 /bin/bash -c > /usr/java/jdk1.7.0_25/bin/java -Djava.net.preferIPv4Stack=true > -Dhadoop.metrics.log.level=WARN -Xmx200m > -Djava.io.tmpdir=/tmp/hadoop-general/nm-local-dir/usercache/general/appcache/application_1388632710048_0009/container_1388632710048_0009_01_000004/tmp > -Dlog4j.configuration=container-log4j.properties > -Dyarn.app.container.log.dir=/home/general/hadoop-2.2.0/logs/userlogs/application_1388632710048_0009/container_1388632710048_0009_01_000004 > -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA > org.apache.hadoop.mapred.YarnChild 127.0.0.1 56498 > attempt_1388632710048_0009_m_000000_2 4 > 1>/home/general/hadoop-2.2.0/logs/userlogs/application_1388632710048_0009/container_1388632710048_0009_01_000004/stdout > > 2>/home/general/hadoop-2.2.0/logs/userlogs/application_1388632710048_0009/container_1388632710048_0009_01_000004/stderr > > |- 12075 12018 12013 12013 (phantomjs) 17 1 1615687680 6539 > /usr/local/bin/phantomjs --webdriver=29062 > --webdriver-logfile=/tmp/hadoop-general/nm-local-dir/usercache/general/appcache/application_1388632710048_0009/container_1388632710048_0009_01_000004/phantomjsdriver.log > > |- 12074 12018 12013 12013 (phantomjs) 16 2 1641000960 6727 > /usr/local/bin/phantomjs --webdriver=5958 > --webdriver-logfile=/tmp/hadoop-general/nm-local-dir/usercache/general/appcache/application_1388632710048_0009/container_1388632710048_0009_01_000004/phantomjsdriver.log > > |- 12073 12018 12013 12013 (phantomjs) 17 2 1641000960 6732 > /usr/local/bin/phantomjs --webdriver=31836 > --webdriver-logfile=/tmp/hadoop-general/nm-local-dir/usercache/general/appcache/application_1388632710048_0009/container_1388632710048_0009_01_000004/phantomjsdriver.log > > |- 12090 12018 12013 12013 (phantomjs) 16 2 1615687680 6538 > /usr/local/bin/phantomjs --webdriver=24519 > --webdriver-logfile=/tmp/hadoop-general/nm-local-dir/usercache/general/appcache/application_1388632710048_0009/container_1388632710048_0009_01_000004/phantomjsdriver.log > > |- 12072 12018 12013 12013 (phantomjs) 16 1 1641000960 6216 > /usr/local/bin/phantomjs --webdriver=10175 > --webdriver-logfile=/tmp/hadoop-general/nm-local-dir/usercache/general/appcache/application_1388632710048_0009/container_1388632710048_0009_01_000004/phantomjsdriver.log > > |- 12091 12018 12013 12013 (phantomjs) 17 1 1615687680 6036 > /usr/local/bin/phantomjs --webdriver=5043 > --webdriver-logfile=/tmp/hadoop-general/nm-local-dir/usercache/general/appcache/application_1388632710048_0009/container_1388632710048_0009_01_000004/phantomjsdriver.log > > |- 12018 12013 12013 12013 (java) 996 41 820924416 79595 > /usr/java/jdk1.7.0_25/bin/java -Djava.net.preferIPv4Stack=true > -Dhadoop.metrics.log.level=WARN -Xmx200m > -Djava.io.tmpdir=/tmp/hadoop-general/nm-local-dir/usercache/general/appcache/application_1388632710048_0009/container_1388632710048_0009_01_000004/tmp > -Dlog4j.configuration=container-log4j.properties > -Dyarn.app.container.log.dir=/home/general/hadoop-2.2.0/logs/userlogs/application_1388632710048_0009/container_1388632710048_0009_01_000004 > -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA > org.apache.hadoop.mapred.YarnChild 127.0.0.1 56498 > attempt_1388632710048_0009_m_000000_2 4 > |- 12078 12018 12013 12013 (phantomjs) 16 3 1615687680 6545 > /usr/local/bin/phantomjs --webdriver=12650 > --webdriver-logfile=/tmp/hadoop-general/nm-local-dir/usercache/general/appcache/application_1388632710048_0009/container_1388632710048_0009_01_000004/phantomjsdriver.log > > |- 12079 12018 12013 12013 (phantomjs) 17 2 1642020864 7542 > /usr/local/bin/phantomjs --webdriver=18444 > --webdriver-logfile=/tmp/hadoop-general/nm-local-dir/usercache/general/appcache/application_1388632710048_0009/container_1388632710048_0009_01_000004/phantomjsdriver.log > > > Container killed on request. Exit code is 143 > -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
