Hi,
Below are my yarn configs. I have tried changing/increasing the yarn memory
limits, and also map/reduce memory limits in mapred-site.xml, but it doesn’t
help.
Thank you!
Muntasir.
Yarn-site.xml:
“<configuration>
<!-- Site specific YARN configuration properties -->
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>node-00.pcap-cluster.iss.emulab.net:8025</value>
</property>
<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>node-00.pcap-cluster.iss.emulab.net:8030</value>
</property>
<!-- Use CapacityScheduler -->
<property>
<name>yarn.resourcemanager.scheduler.class</name>
<value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler</value>
</property>
<property>
<name>yarn.resourcemanager.address</name>
<value>node-00.pcap-cluster.iss.emulab.net:8040</value>
</property>
<property>
<name>yarn.nodemanager.local-dirs</name>
<value>/tmp/mrahman2/hadoop-2.7.0/nm-local-dirs</value>
</property>
<property>
<name>yarn.nodemanager.log-dirs</name>
<value>/tmp/mrahman2/hadoop-2.7.0/nm-log-dirs</value>
</property>
<property>
<name>yarn.nodemanager.vmem-check-enabled</name>
<value>false</value>
</property>
<property>
<name>yarn.nodemanager.pmem-check-enabled</name>
<value>false</value>
</property>
<property>
<name>yarn.scheduler.minimum-allocation-mb</name>
<value>4096</value>
</property>
<property>
<name>yarn.scheduler.maximum-allocation-mb</name>
<value>8192</value>
</property>
<property>
<name>yarn.resourcemanager.resume-local-only</name>
<value>false</value>
</property>
<!-- medium containers -->
<property>
<name>yarn.nodemanager.resource.memory-mb</name>
<value>4096</value>
</property>
</configuration>
”
From yarn-env.sh:
“export HADOOP_HOME=/mnt/hadoop/hadoop-2.7.0
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export YARN_HOME=$HADOOP_HOME
# User for YARN daemons
export HADOOP_YARN_USER=${HADOOP_YARN_USER:-yarn}
# resolve links - $0 may be a softlink
export YARN_CONF_DIR="${YARN_CONF_DIR:-$YARN_HOME/conf}"
export HADOOP_CONF_DIR=${YARN_CONF_DIR}
# some Java parameters
# export JAVA_HOME=/home/y/libexec/jdk1.6.0/
export JAVA_HOME=/usr
echo $JAVA_HOME
if [ "$JAVA_HOME" != "" ]; then
#echo "run java in $JAVA_HOME"
JAVA_HOME=$JAVA_HOME
fi
if [ "$JAVA_HOME" = "" ]; then
echo "Error: JAVA_HOME is not set."
exit 1
fi
JAVA=$JAVA_HOME/bin/java
# small containers
JAVA_HEAP_MAX=-Xmx500m
if [ "$YARN_HEAPSIZE" != "" ]; then
#echo "run with heapsize $YARN_HEAPSIZE"
JAVA_HEAP_MAX="-Xmx""$YARN_HEAPSIZE""m"
#echo $JAVA_HEAP_MAX
Fi
IFS=
# default log directory & file
if [ "$YARN_LOG_DIR" = "" ]; then
YARN_LOG_DIR="/tmp/mrahman2/hadoop-2.7.0/yarn-logs"
fi
if [ "$YARN_LOGFILE" = "" ]; then
YARN_LOGFILE='yarn.log'
fi
# default policy file for service-level authorization
if [ "$YARN_POLICYFILE" = "" ]; then
YARN_POLICYFILE="hadoop-policy.xml"
fi
# restore ordinary behaviour
unset IFS
YARN_OPTS="$YARN_OPTS -Dhadoop.log.dir=$YARN_LOG_DIR"
YARN_OPTS="$YARN_OPTS -Dyarn.log.dir=$YARN_LOG_DIR"
YARN_OPTS="$YARN_OPTS -Dhadoop.log.file=$YARN_LOGFILE"
YARN_OPTS="$YARN_OPTS -Dyarn.log.file=$YARN_LOGFILE"
YARN_OPTS="$YARN_OPTS -Dyarn.home.dir=$YARN_COMMON_HOME"
YARN_OPTS="$YARN_OPTS -Dyarn.id.str=$YARN_IDENT_STRING"
YARN_OPTS="$YARN_OPTS -Dhadoop.root.logger=${YARN_ROOT_LOGGER:-INFO,console}"
YARN_OPTS="$YARN_OPTS -Dyarn.root.logger=${YARN_ROOT_LOGGER:-INFO,console}"
if [ "x$JAVA_LIBRARY_PATH" != "x" ]; then
YARN_OPTS="$YARN_OPTS -Djava.library.path=$JAVA_LIBRARY_PATH"
fi
YARN_OPTS="$YARN_OPTS -Dyarn.policy.file=$YARN_POLICYFILE"
”
From: Ramesh Krishnan [mailto:[email protected]]
Sent: Sunday, May 22, 2016 11:31 AM
To: [email protected]
Subject: Re: container failure while running Giraph Shortest Path on Yarn
Hi,
This does not seem to be a dependecy issue . This is an exception in launch of
the container. Can you check the yarn configurations.
Thanks,
Ramesh
On Sun, May 22, 2016 at 9:52 PM, Rahman, Muntasir Raihan
<[email protected]<mailto:[email protected]>> wrote:
Thanks for the suggestion, but unfortunately it didn’t fix the issue.
Muntasir.
From: Ravikant Dindokar
[mailto:[email protected]<mailto:[email protected]>]
Sent: Sunday, May 22, 2016 3:37 AM
To: [email protected]<mailto:[email protected]>
Subject: Re: container failure while running Giraph Shortest Path on Yarn
try adding
-libjars
giraph-core/target/giraph-1.1.0-for-hadoop-2.7.0-jar-with-dependencies.jar
after org.apache.giraph.GiraphRunner
On Sat, May 21, 2016 at 4:21 AM, Rahman, Muntasir Raihan
<[email protected]<mailto:[email protected]>> wrote:
Hi,
I have been trying to run some Giraph programs on top of Yarn, but I keep
getting the following error:
“Container id: container_1463763127358_0002_01_000003
Exit code: 2
Stack trace: ExitCodeException exitCode=2:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:545)
at org.apache.hadoop.util.Shell.run(Shell.java:456)
at
org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722)
at
org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Container exited with a non-zero exit code 2
”
I ran the following command:
“$HADOOP_HOME/bin/hadoop jar
giraph-examples/target/giraph-examples-1.1.0-for-hadoop-2.7.0-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner
org.apache.giraph.examples.SimpleShortestPathsComputation -vif
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip
<inputPath> -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
<outputPath> -w 1 -yj
giraph-examples-1.1.0-for-hadoop-2.7.0-jar-with-dependencies.jar”
Are there any suggestions or hints to fix this?
Regards
Muntasir.
