Hi, This is the content of my shell script for running the job history server:
cd $HADOOP_PREFIX hadoop fs -mkdir -p /mr-history/tmp hadoop fs -chmod -R 1777 /mr-history/tmp hadoop fs -mkdir -p /mr-history/done hadoop fs -chmod -R 1777 /mr-history/done sbin/mr-jobhistory-daemon.sh start historyserver These configurable variables are in mapred-site.xml <property> <name>mapreduce.jobhistory.address</name> <value>137.195.143.129:10020</value> <description>Default port is 10020.</description> </property> <property> <name>mapreduce.jobhistory.webapp.address</name> <value>137.195.143.129:19888</value> <description>Default port is 19888.</description> </property> I start the history server on the same node as my resource manager The counters are available from when the job is running from: http://your-server:8088/proxy/application_1408007466921_0002/mapreduce/job/ job_1408007466921_0002 Drill down through the application master to the job. If you donĀ¹t have the history server running the job data is not persistent. Hope this helps. Regards, Chris MacKenzie telephone: 0131 332 6967 email: stu...@chrismackenziephotography.co.uk corporate: www.chrismackenziephotography.co.uk <http://www.chrismackenziephotography.co.uk/> <http://plus.google.com/+ChrismackenziephotographyCoUk/posts> <http://www.linkedin.com/in/chrismackenziephotography/> From: ou senshaw <sens...@gmail.com> Reply-To: <user@hadoop.apache.org> Date: Thursday, 14 August 2014 07:14 To: <user@hadoop.apache.org> Subject: Hadoop 2.2 Built-in Counters Hi all, I'm trying to analyze my mapreduce job performance via built-in counters such as physical memory usage, heap memory usage... When the job is running, I can watch these counters via Resource manager website(namenode:8088). However, when the job is done, counter information is not available in resource manager website anymore. I know I can get them from client output. I was wondering if there is other place in name node or data node to get the final counter measures regarding job id? Thanks, Shaw