Instead of localhost, in the servers block, we now put the machine that
has gmetad running.
dfs.class=org.apache.hadoop.metrics.ganglia.GangliaContext
dfs.period=10
dfs.servers=GMETAD_HOST:8649
Jason Venner wrote:
Well, with the metrics file changed to perform file based logging,
metrics do appear.
On digging into the GangliaContext source, it looks like it is using
udp for reporting, and we modified the gmond.conf to receive via udp
as well as tcp. netstat -a -p shows gmond monitoring 8649 for both tcp
and udp.
Still nothing visible via the ganglia ui and no rrd file for anything
hadoop related.
Jason Venner wrote:
We have modified my metrics file, distributed it and restarted our
cluster. We have gmond running on the nodes, and a machine on the
vlan with gmetad running.
We have statistics for the machines in the web ui, and our statistics
reported by the gmetric program are present. We don't see any hadoop
reporting.
Clearly we have something basic wrong in our understanding of how to
set this up.
# Configuration of the "dfs" context for null
# dfs.class=org.apache.hadoop.metrics.spi.NullContext
# Configuration of the "dfs" context for file
#dfs.class=org.apache.hadoop.metrics.file.FileContext
#dfs.period=10
#dfs.fileName=/tmp/dfsmetrics.log
# Configuration of the "dfs" context for ganglia
dfs.class=org.apache.hadoop.metrics.ganglia.GangliaContext
dfs.period=10
dfs.servers=localhost:8649
# Configuration of the "mapred" context for null
# mapred.class=org.apache.hadoop.metrics.spi.NullContext
# Configuration of the "mapred" context for file
#mapred.class=org.apache.hadoop.metrics.file.FileContext
#mapred.period=10
#mapred.fileName=/tmp/mrmetrics.log
# Configuration of the "mapred" context for ganglia
mapred.class=org.apache.hadoop.metrics.ganglia.GangliaContext
mapred.period=10
mapred.servers=localhost:8649
# Configuration of the "jvm" context for null
# jvm.class=org.apache.hadoop.metrics.spi.NullContext
# Configuration of the "jvm" context for file
#jvm.class=org.apache.hadoop.metrics.file.FileContext
#jvm.period=10
#jvm.fileName=/tmp/jvmmetrics.log
# Configuration of the "jvm" context for ganglia
jvm.class=org.apache.hadoop.metrics.ganglia.GangliaContext
jvm.period=10
jvm.servers=localhost:8649
--
Jason Venner
Attributor - Publish with Confidence <http://www.attributor.com/>
Attributor is hiring Hadoop Wranglers, contact if interested
