never mind on the jvm. just found the typo.. <frown>
On Mar 6, 2008, at 3:19 PM, Chris K Wensel wrote:
actually, I don't see any jvm metrics across the cluster.
any idea how to get a local gmond to gmetad to report local
statistics? it is also accumulating slave stats just fine (minus jvm).
On Mar 6, 2008, at 3:09 PM, Jason Venner wrote:
We have tweeked our metrics properties to cause all nodes to report
to the master node, and we see the map/reduce metrics also see the
jvm metrics from all nodes.
The other thing is that this only runs on the master node, as it is
only run from the driver which currently runs on the master node.
As I look deeper, I believe the metrics are not being flushed when
the context is shut down, and the flush methods are not implemented
for the ganglia context.
Chris K Wensel wrote:
I have ganglia up on my cluster, and I definitely see some metrics
from the map/reduce tasks. But I don't see anything from the JVM
context for ganglia except from the master node.
but i have gmetad running on the master and reporting from a local
gmond seems flakey.
On Mar 6, 2008, at 2:03 PM, Jason Venner wrote:
We have started our first attempt at this, and do not see the
metrics being reported.
Our first cut simply is trying to report the counters at the end
of the job.
A theory is that the job is exiting before the metrics are flushed.
This code is in the driver for our map/reduce task, and is called
after the JobClient.runJob() method returns.
private ContextFactory contextFactory = null;
private MetricsContext myContext = null;
private MetricsRecord randomizeRecord = null;
{
try {
contextFactory = ContextFactory.getFactory();
myContext = contextFactory.getContext("myContext");
randomizeRecord =
myContext.createRecord( "MyContextName" );
} catch( Exception e ) {
LOG.error( "Unable to initialize metrics context
factory", e );
}
}
void reportCounterMetrics( Counters counters ) {
for( String groupName : counters.getGroupNames() ) {
Group group = counters.getGroup( groupName );
for( String counterName : group.getCounterNames() ) {
randomizeRecord.setMetric( counterName,
group.getCounter(counterName) );
LOG.info( "reporting counter " + counterName + " " +
group.getCounter(counterName) );
}
}
randomizeRecord.update();
}
Chris K Wensel
[EMAIL PROTECTED]
http://chris.wensel.net/
Chris K Wensel
[EMAIL PROTECTED]
http://chris.wensel.net/
Chris K Wensel
[EMAIL PROTECTED]
http://chris.wensel.net/
