Job stuck in running state on Hadoop 2.2.0

Hi everyone, I'm having trouble running the Hadoop examples in a single node. All the executions get stuck at the running state at 0% map and reduce and the logs don't seem to indicate any issue, besides the need to kill the node manager: compute-0-7-3: nodemanager did not stop gracefully after 5

Re: Job stuck in running state on Hadoop 2.2.0

I had a similar problem after setting up Hadoop 2.2.0 based on the instructions at http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/SingleCluster.html Although it's not documented on the page, I needed to edit hadoop-env.sh and yarn-env.sh as well to update JAVA_HOME, HADOOP

Re: Job stuck in running state on Hadoop 2.2.0

Thank you! I realized that, despite I exported the variables in the scripts, there were a few errors and my desired configuration wasn't being used (which explained other strange behavior). However, I'm still getting the same issue with the examples, for instance: hadoop jar ~/hadoop-2.2.0-maven/

Re: Job stuck in running state on Hadoop 2.2.0

It sounds like the job was successfully submitted to the cluster, but there as some problem when starting/running AM, so that no progress is made. It happened to me once, when I was playing with YARN on a cluster consisting of very small machines, and I mis-configured YARN to allocated to AM more m

Re: Job stuck in running state on Hadoop 2.2.0

Actually now it seems to be running (or at least attempting to run) but I get further errors: hadoop jar ~/hadoop-2.2.0-maven/hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.0.0-SNAPSHOT.jar pi 1 100 INFO mapreduce.Job: Job job_1386751964857_0001 failed

Re: Job stuck in running state on Hadoop 2.2.0

OK that was indeed a classpath issue, which I solved by directly exporting the output of hadoop classpath (ie. the list of neeed jars, see this) into HADOOP_CLASSPATH in hadoop-env.sh and yarn-env.sh With this fixed, the stuck issue came back so I

Re: Job stuck in running state on Hadoop 2.2.0

I checked yarn-site.xml configuration and I tried to run the program without the memory configurations I found somewhere and assumed that would work (yarn.nodemanager.resource.memory-mb=2200 and yarn.scheduler.minimum-allocation-mb=500) following Adam's advice and the example worked beautifully :D

Re: Job stuck in running state on Hadoop 2.2.0

I am glad that I could help. In our case, we followed mostly the configuration from here: http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/ClusterSetup.html (changing it a bit to adapt to our requirements e.g. today we run 2GB containers instead of 3-4GB, but it might change