Hi Dino
It can't be used effectively to monitor MR mappers since tasks on a node
run as separate JVMs and each task will overwrite the same metric
published by other tasks. You cannot aggregate values from running tasks
before publish them because tasks don't communicate with each other.
If you publish a metric per task, this will give you tones of graphs in
Ganglia.
On other hand I extended it the API and use it to monitor our client
(webapp) and it works fine and provide valuable information. I have
started a blog post to document this, will be ready soon, I'll add the
link here if you need some examples. I wonder why this API is not a
separate project, it can be reused in apps that don't need Hadoop. I
could not find any java client API to publish metrics in Ganglia.
Regards
Daniel
On 11/02/2011 09:44 AM, Dino Kečo wrote:
Hi Hadoop users,
I have been reading about Hadoop metrics framework, and I was
wondering is it possible to create custom metrics for specific job.
In my use case I want to capture some specific information per record
which is processed. So I was thinking to create custom code in my
Hadoop job to send these information using Hadoop metrics. Is this
possible to implement ? If so can anyone share any example how things
should be done?
Thanks,
dinok
