Re: Hadoop and Ganglia Meterics
Ah, yeah, I found that one. :) Patching 'java/org/apache/hadoop/mapred/JobInProgress.java' on 0.17.1. -joe Jason Venner wrote: I have only applied this patch as far forward as 0.16.0 Joe Williams wrote: Sweet, thanks. Jason Venner wrote: Once the patch is applied you should start seeing the ganglia metrics We do. Joe Williams wrote: Once I have the patch applied and have it running should I see the metrics? Or do I need to additional work? Thanks. -Joe Jason Venner wrote: I applied the patch in the jira to my distro Joe Williams wrote: Thanks Jason, until this is implemented are how are you pulling stats from Hadoop? -joe Jason Venner wrote: Check out https://issues.apache.org/jira/browse/HADOOP-3422 Joe Williams wrote: I have been attempting to get Hadoop metrics in Ganliga and have been unsuccessful thus far. I have see this thread (http://mail-archives.apache.org/mod_mbox/hadoop-core-user/200712.mbox/raw/[EMAIL PROTECTED]/) but it didn't help much. I have setup my properties file like so: [EMAIL PROTECTED] current]# cat conf/hadoop-metrics.properties dfs.class=org.apache.hadoop.metrics.ganglia.GangliaContext dfs.period=10 dfs.servers=127.0.0.1:8649 mapred.class=org.apache.hadoop.metrics.ganglia.GangliaContext mapred.period=10 mapred.servers=127.0.0.1:8649 And if I 'telnet 127.0.0.1 8649' I receive the Ganglia XML metrics output without any hadoop specific metrics: [EMAIL PROTECTED] current]# telnet 127.0.0.1 8649 Trying 127.0.0.1... Connected to localhost (127.0.0.1). Escape character is '^]'. ?xml version="1.0" encoding="ISO-8859-1" standalone="yes"?> !DOCTYPE GANGLIA_XML [ !ELEMENT GANGLIA_XML (GRID|CLUSTER|HOST)*> !ATTLIST GANGLIA_XML VERSION CDATA #REQUIRED> !ATTLIST GANGLIA_XML SOURCE CDATA #REQUIRED> --SNIP-- Is there more I need to do to get the metrics to show up in this output, am I doing something incorrectly? Do I need to have a gmetric script run in a cron to update the stats? If so, does anyone have a hadoop specific example of this? Any info would be helpful. Thanks. -Joe -- Name: Joseph A. Williams Email: [EMAIL PROTECTED]
Re: Hadoop and Ganglia Meterics
Sweet, thanks. Jason Venner wrote: Once the patch is applied you should start seeing the ganglia metrics We do. Joe Williams wrote: Once I have the patch applied and have it running should I see the metrics? Or do I need to additional work? Thanks. -Joe Jason Venner wrote: I applied the patch in the jira to my distro Joe Williams wrote: Thanks Jason, until this is implemented are how are you pulling stats from Hadoop? -joe Jason Venner wrote: Check out https://issues.apache.org/jira/browse/HADOOP-3422 Joe Williams wrote: I have been attempting to get Hadoop metrics in Ganliga and have been unsuccessful thus far. I have see this thread (http://mail-archives.apache.org/mod_mbox/hadoop-core-user/200712.mbox/raw/[EMAIL PROTECTED]/) but it didn't help much. I have setup my properties file like so: [EMAIL PROTECTED] current]# cat conf/hadoop-metrics.properties dfs.class=org.apache.hadoop.metrics.ganglia.GangliaContext dfs.period=10 dfs.servers=127.0.0.1:8649 mapred.class=org.apache.hadoop.metrics.ganglia.GangliaContext mapred.period=10 mapred.servers=127.0.0.1:8649 And if I 'telnet 127.0.0.1 8649' I receive the Ganglia XML metrics output without any hadoop specific metrics: [EMAIL PROTECTED] current]# telnet 127.0.0.1 8649 Trying 127.0.0.1... Connected to localhost (127.0.0.1). Escape character is '^]'. ?xml version="1.0" encoding="ISO-8859-1" standalone="yes"?> !DOCTYPE GANGLIA_XML [ !ELEMENT GANGLIA_XML (GRID|CLUSTER|HOST)*> !ATTLIST GANGLIA_XML VERSION CDATA #REQUIRED> !ATTLIST GANGLIA_XML SOURCE CDATA #REQUIRED> --SNIP-- Is there more I need to do to get the metrics to show up in this output, am I doing something incorrectly? Do I need to have a gmetric script run in a cron to update the stats? If so, does anyone have a hadoop specific example of this? Any info would be helpful. Thanks. -Joe -- Name: Joseph A. Williams Email: [EMAIL PROTECTED]
Re: Hadoop and Ganglia Meterics
Once I have the patch applied and have it running should I see the metrics? Or do I need to additional work? Thanks. -Joe Jason Venner wrote: I applied the patch in the jira to my distro Joe Williams wrote: Thanks Jason, until this is implemented are how are you pulling stats from Hadoop? -joe Jason Venner wrote: Check out https://issues.apache.org/jira/browse/HADOOP-3422 Joe Williams wrote: I have been attempting to get Hadoop metrics in Ganliga and have been unsuccessful thus far. I have see this thread (http://mail-archives.apache.org/mod_mbox/hadoop-core-user/200712.mbox/raw/[EMAIL PROTECTED]/) but it didn't help much. I have setup my properties file like so: [EMAIL PROTECTED] current]# cat conf/hadoop-metrics.properties dfs.class=org.apache.hadoop.metrics.ganglia.GangliaContext dfs.period=10 dfs.servers=127.0.0.1:8649 mapred.class=org.apache.hadoop.metrics.ganglia.GangliaContext mapred.period=10 mapred.servers=127.0.0.1:8649 And if I 'telnet 127.0.0.1 8649' I receive the Ganglia XML metrics output without any hadoop specific metrics: [EMAIL PROTECTED] current]# telnet 127.0.0.1 8649 Trying 127.0.0.1... Connected to localhost (127.0.0.1). Escape character is '^]'. ?xml version="1.0" encoding="ISO-8859-1" standalone="yes"?> !DOCTYPE GANGLIA_XML [ !ELEMENT GANGLIA_XML (GRID|CLUSTER|HOST)*> !ATTLIST GANGLIA_XML VERSION CDATA #REQUIRED> !ATTLIST GANGLIA_XML SOURCE CDATA #REQUIRED> --SNIP-- Is there more I need to do to get the metrics to show up in this output, am I doing something incorrectly? Do I need to have a gmetric script run in a cron to update the stats? If so, does anyone have a hadoop specific example of this? Any info would be helpful. Thanks. -Joe -- Name: Joseph A. Williams Email: [EMAIL PROTECTED]
Re: Hadoop and Ganglia Meterics
Thanks Jason, until this is implemented are how are you pulling stats from Hadoop? -joe Jason Venner wrote: Check out https://issues.apache.org/jira/browse/HADOOP-3422 Joe Williams wrote: I have been attempting to get Hadoop metrics in Ganliga and have been unsuccessful thus far. I have see this thread (http://mail-archives.apache.org/mod_mbox/hadoop-core-user/200712.mbox/raw/[EMAIL PROTECTED]/) but it didn't help much. I have setup my properties file like so: [EMAIL PROTECTED] current]# cat conf/hadoop-metrics.properties dfs.class=org.apache.hadoop.metrics.ganglia.GangliaContext dfs.period=10 dfs.servers=127.0.0.1:8649 mapred.class=org.apache.hadoop.metrics.ganglia.GangliaContext mapred.period=10 mapred.servers=127.0.0.1:8649 And if I 'telnet 127.0.0.1 8649' I receive the Ganglia XML metrics output without any hadoop specific metrics: [EMAIL PROTECTED] current]# telnet 127.0.0.1 8649 Trying 127.0.0.1... Connected to localhost (127.0.0.1). Escape character is '^]'. ?xml version="1.0" encoding="ISO-8859-1" standalone="yes"?> !DOCTYPE GANGLIA_XML [ !ELEMENT GANGLIA_XML (GRID|CLUSTER|HOST)*> !ATTLIST GANGLIA_XML VERSION CDATA #REQUIRED> !ATTLIST GANGLIA_XML SOURCE CDATA #REQUIRED> --SNIP-- Is there more I need to do to get the metrics to show up in this output, am I doing something incorrectly? Do I need to have a gmetric script run in a cron to update the stats? If so, does anyone have a hadoop specific example of this? Any info would be helpful. Thanks. -Joe -- Name: Joseph A. Williams Email: [EMAIL PROTECTED]
Hadoop and Ganglia Meterics
I have been attempting to get Hadoop metrics in Ganliga and have been unsuccessful thus far. I have see this thread (http://mail-archives.apache.org/mod_mbox/hadoop-core-user/200712.mbox/raw/[EMAIL PROTECTED]/) but it didn't help much. I have setup my properties file like so: [EMAIL PROTECTED] current]# cat conf/hadoop-metrics.properties dfs.class=org.apache.hadoop.metrics.ganglia.GangliaContext dfs.period=10 dfs.servers=127.0.0.1:8649 mapred.class=org.apache.hadoop.metrics.ganglia.GangliaContext mapred.period=10 mapred.servers=127.0.0.1:8649 And if I 'telnet 127.0.0.1 8649' I receive the Ganglia XML metrics output without any hadoop specific metrics: [EMAIL PROTECTED] current]# telnet 127.0.0.1 8649 Trying 127.0.0.1... Connected to localhost (127.0.0.1). Escape character is '^]'. ?xml version="1.0" encoding="ISO-8859-1" standalone="yes"?> !DOCTYPE GANGLIA_XML [ !ELEMENT GANGLIA_XML (GRID|CLUSTER|HOST)*> !ATTLIST GANGLIA_XML VERSION CDATA #REQUIRED> !ATTLIST GANGLIA_XML SOURCE CDATA #REQUIRED> --SNIP-- Is there more I need to do to get the metrics to show up in this output, am I doing something incorrectly? Do I need to have a gmetric script run in a cron to update the stats? If so, does anyone have a hadoop specific example of this? Any info would be helpful. Thanks. -Joe -- Name: Joseph A. Williams Email: [EMAIL PROTECTED]
