Thanks guys! It's working.
On Thu, Aug 14, 2014 at 2:39 AM, Chris MacKenzie
stu...@chrismackenziephotography.co.uk wrote:
Hi,
This is the content of my shell script for running the job history server:
cd $HADOOP_PREFIX
hadoop fs -mkdir -p /mr-history/tmp
hadoop fs -chmod -R 1777 /mr-history/tmp
hadoop fs -mkdir -p /mr-history/done
hadoop fs -chmod -R 1777 /mr-history/done
sbin/mr-jobhistory-daemon.sh start historyserver
These configurable variables are in mapred-site.xml
property
namemapreduce.jobhistory.address/name
value137.195.143.129:10020/value
descriptionDefault port is 10020./description
/property
property
namemapreduce.jobhistory.webapp.address/name
value137.195.143.129:19888/value
descriptionDefault port is 19888./description
/property
I start the history server on the same node as my resource manager
The counters are available from when the job is running from:
http://your-server:8088/proxy/application_1408007466921_0002/mapreduce/job/
job_1408007466921_0002
Drill down through the application master to the job.
If you don¹t have the history server running the job data is not
persistent.
Hope this helps.
Regards,
Chris MacKenzie
telephone: 0131 332 6967
email: stu...@chrismackenziephotography.co.uk
corporate: www.chrismackenziephotography.co.uk
http://www.chrismackenziephotography.co.uk/
http://plus.google.com/+ChrismackenziephotographyCoUk/posts
http://www.linkedin.com/in/chrismackenziephotography/
From: ou senshaw sens...@gmail.com
Reply-To: user@hadoop.apache.org
Date: Thursday, 14 August 2014 07:14
To: user@hadoop.apache.org
Subject: Hadoop 2.2 Built-in Counters
Hi all,
I'm trying to analyze my mapreduce job performance via built-in counters
such as physical memory usage, heap memory usage...
When the job is running, I can watch these counters via Resource manager
website(namenode:8088). However, when the job is done, counter information
is not available in resource manager website anymore. I know I can get
them from client output. I was wondering if there is other place in name
node or data node to get the final counter measures regarding job id?
Thanks,
Shaw
