Hi, 
I have set up a single node Hadoop 2.2.0 .
I can submit  wordcount  hadoop job to YARN from the command line.
It run successfully and I checked the result is in hdfs. 
matmsh@gauss:~> hadoop jar  
~/installed/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar  
wordcount input output6
14/06/28 13:40:00 WARN util.NativeCodeLoader: Unable to load native-hadoop 
library for your platform... using builtin-java classes where applicable
14/06/28 13:40:00 INFO Configuration.deprecation: session.id is deprecated. 
Instead, use dfs.metrics.session-id
14/06/28 13:40:00 INFO jvm.JvmMetrics: Initializing JVM Metrics with 
processName=JobTracker, sessionId=
14/06/28 13:40:00 INFO input.FileInputFormat: Total input paths to process : 1
14/06/28 13:40:00 INFO mapreduce.JobSubmitter: number of splits:1
14/06/28 13:40:00 INFO Configuration.deprecation: user.name is deprecated. 
Instead, use mapreduce.job.user.name
14/06/28 13:40:00 INFO Configuration.deprecation: mapred.jar is deprecated. 
Instead, use mapreduce.job.jar
14/06/28 13:40:00 INFO Configuration.deprecation: mapred.output.value.class is 
deprecated. Instead, use mapreduce.job.output.value.class
14/06/28 13:40:00 INFO Configuration.deprecation: mapreduce.combine.class is 
deprecated. Instead, use mapreduce.job.combine.class
14/06/28 13:40:00 INFO Configuration.deprecation: mapreduce.map.class is 
deprecated. Instead, use mapreduce.job.map.class
14/06/28 13:40:00 INFO Configuration.deprecation: mapred.job.name is 
deprecated. Instead, use mapreduce.job.name
14/06/28 13:40:00 INFO Configuration.deprecation: mapreduce.reduce.class is 
deprecated. Instead, use mapreduce.job.reduce.class
14/06/28 13:40:00 INFO Configuration.deprecation: mapred.input.dir is 
deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
14/06/28 13:40:00 INFO Configuration.deprecation: mapred.output.dir is 
deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
14/06/28 13:40:00 INFO Configuration.deprecation: mapred.map.tasks is 
deprecated. Instead, use mapreduce.job.maps
14/06/28 13:40:00 INFO Configuration.deprecation: mapred.output.key.class is 
deprecated. Instead, use mapreduce.job.output.key.class
14/06/28 13:40:00 INFO Configuration.deprecation: mapred.working.dir is 
deprecated. Instead, use mapreduce.job.working.dir
14/06/28 13:40:00 INFO mapreduce.JobSubmitter: Submitting tokens for job: 
job_local28288787_0001
14/06/28 13:40:00 WARN conf.Configuration: 
file:/tmp/hadoop-matmsh/mapred/staging/matmsh28288787/.staging/job_local28288787_0001/job.xml:an
 attempt to override final parameter: 
mapreduce.job.end-notification.max.retry.interval;  Ignoring.
14/06/28 13:40:00 WARN conf.Configuration: 
file:/tmp/hadoop-matmsh/mapred/staging/matmsh28288787/.staging/job_local28288787_0001/job.xml:an
 attempt to override final parameter: 
mapreduce.job.end-notification.max.attempts;  Ignoring.
14/06/28 13:40:00 WARN conf.Configuration: 
file:/tmp/hadoop-matmsh/mapred/local/localRunner/matmsh/job_local28288787_0001/job_local28288787_0001.xml:an
 attempt to override final parameter: 
mapreduce.job.end-notification.max.retry.interval;  Ignoring.
14/06/28 13:40:00 WARN conf.Configuration: 
file:/tmp/hadoop-matmsh/mapred/local/localRunner/matmsh/job_local28288787_0001/job_local28288787_0001.xml:an
 attempt to override final parameter: 
mapreduce.job.end-notification.max.attempts;  Ignoring.
14/06/28 13:40:00 INFO mapreduce.Job: The url to track the job: 
http://localhost:8080/
14/06/28 13:40:00 INFO mapreduce.Job: Running job: job_local28288787_0001
14/06/28 13:40:00 INFO mapred.LocalJobRunner: OutputCommitter set in config null
14/06/28 13:40:00 INFO mapred.LocalJobRunner: OutputCommitter is 
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
14/06/28 13:40:01 INFO mapred.LocalJobRunner: Waiting for map tasks
14/06/28 13:40:01 INFO mapred.LocalJobRunner: Starting task: 
attempt_local28288787_0001_m_000000_0
14/06/28 13:40:01 INFO mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
14/06/28 13:40:01 INFO mapred.MapTask: Processing split: 
hdfs://localhost:9000/user/matmsh/input/getdoc.txt:0+23992
14/06/28 13:40:01 INFO mapred.MapTask: Map output collector class = 
org.apache.hadoop.mapred.MapTask$MapOutputBuffer
14/06/28 13:40:01 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
.....

However, the above job does not show up at  the web UI :
http://localhost:8088/cluster/apps

My configurations are below, 
mapred-site.xml :
<configuration>

<property>
<value>mapreduce.framework.name</value>
<name>yarn</name>
</property>

</configuration>

-------
hdfs-site.xml


<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>

<property>
<name>dfs.namenode.name.dir</name>
<value>file:/home/matmsh/temp/hadoop2/namenode</value>
<description>The name of the default file system.</description>
</property>

<property>
<name>dfs.datanode.data.dir</name>
<value>file:/home/matmsh/temp/hadoop2/datanode</value>
<description>The name of the default file system.</description>
</property> 

</configuration>
---------------- 
yarn-site.xml 
<configuration>

<!-- Site specific YARN configuration properties -->

<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>


</configuration>

------------------------------------

What other configurations are  needed to show the job in the web UI ? 
Thanks in advance for any  assistance !

Shing 

Reply via email to