Hi all,
m using apache hadoop 1.0.0,centos 6.2 x86_64,ganglia : 3.1.7,ganglia web
3.3
i can get time.string and constant metrics but i couldn't get hadoop
metrics.ganglia 3.1.7 supports apache hadoop 1.0.2?? Below is the
hadoop-metrics2.properties how should i edit this file??
# syntax
Hi Varun Kapoor,
I've been trying to debug but unfortunately, I get the No stack.
message from the gdb.
Here're the details:
* Hadoop: 1.0
* Ganglia: 3.1.7
# gdb /usr/sbin/gmetad
GNU gdb (GDB) CentOS (7.0.1-42.el5.centos)
Copyright (C) 2009 Free Software Foundation, Inc
option..
On 15 February 2012 19:53, mete efk...@gmail.com wrote:
Well rebuilding ganglia seemed easier and Merto was testing the other so i
though that i should give that one a chance :)
anyway i will send you gdb details or patch hadoop and try it at my
earliest convenience
Cheers
On Wed, Feb
which metric's value caused
this buffer overflow) - if you're not, I'll try and send you some gdb
scripts to narrow things down once I see the output from this round of
debugging.
Also, out of curiosity, is patching Hadoop not an option for you? Or is it
just that rebuilding (and redeploying) ganglia
Well rebuilding ganglia seemed easier and Merto was testing the other so i
though that i should give that one a chance :)
anyway i will send you gdb details or patch hadoop and try it at my
earliest convenience
Cheers
On Wed, Feb 15, 2012 at 6:59 PM, Varun Kapoor rez...@hortonworks.comwrote
Hello Varun,
i have patched and recompiled ganglia from source bit it still cores after
the patch.
Here are some logs:
Feb 15 09:39:14 master gmetad[16487]: RRD_update
(/var/lib/ganglia/rrds/hadoop/slave4/metricssystem.MetricsSystem.publish_max_time.rrd):
/var/lib/ganglia/rrds/hadoop/slave4
crashing..
- When you start gmetad and Hadoop is not emitting metrics,
everything
is peachy.
Right, running just ganglia without running hadoop jobs seems
stable
for at
least a day..
- When you start Hadoop (and it thus starts emitting
just ganglia without running hadoop jobs seems
stable
for at
least a day..
- When you start Hadoop (and it thus starts emitting
metrics),
gmetad
cores.
True, with a following error : *** stack smashing detected
***:
gmetad
Mete
On Tue, Feb 7, 2012 at 4:58 AM, Merto Mertek masmer...@gmail.com
wrote:
I have tried to run it but it repeats crashing..
- When you start gmetad and Hadoop is not emitting metrics,
everything
is peachy.
Right, running just ganglia without running
to run it but it repeats crashing..
- When you start gmetad and Hadoop is not emitting metrics,
everything
is peachy.
Right, running just ganglia without running hadoop jobs seems
stable
for at
least a day..
- When you start Hadoop
is smooth.
Regards
Mete
On Tue, Feb 7, 2012 at 4:58 AM, Merto Mertek masmer...@gmail.com wrote:
I have tried to run it but it repeats crashing..
- When you start gmetad and Hadoop is not emitting metrics, everything
is peachy.
Right, running just ganglia without running
, 2012 at 4:58 AM, Merto Mertek masmer...@gmail.com wrote:
I have tried to run it but it repeats crashing..
- When you start gmetad and Hadoop is not emitting metrics, everything
is peachy.
Right, running just ganglia without running hadoop jobs seems stable
for at
least a day
wrote:
I have tried to run it but it repeats crashing..
- When you start gmetad and Hadoop is not emitting metrics,
everything
is peachy.
Right, running just ganglia without running hadoop jobs seems stable
for at
least a day..
- When you start Hadoop
I assume you have seen the following information on Hadoop twiki,
http://wiki.apache.org/hadoop/GangliaMetrics
So do you use GangliaContext31 in hadoop-metrics2.properties?
We use Ganglia 3.2 with Hadoop 20.205 and works fine (I remember seeing
gmetad sometime goes down due to buffer overflow
Hello,
i also face this issue when using GangliaContext31 and hadoop-1.0.0, and
ganglia 3.1.7 (also tried 3.1.2). I continuously get buffer overflows as
soon as i restart the gmetad.
Regards
Mete
On Mon, Feb 6, 2012 at 7:42 PM, Vitthal Suhas Gogate
gog...@hortonworks.com wrote:
I assume you
:7f7762428fdb
sp:7f776362d370 error:0 in libgcc_s.so.1[7f776241a000+15000]
When I compiled gmetad I used the following command:
./configure --with-gmetad --sysconfdir=/etc/ganglia
CPPFLAGS=-I/usr/local/rrdtool-1.4.7/include
CFLAGS=-I/usr/local/rrdtool-1.4.7/include
LDFLAGS=-L/usr/local/rrdtool
Hey Merto,
I've been digging into this problem since Sunday, and believe I may have
root-caused it.
I'm using ganglia-3.2.0, rrdtool-1.4.5 and
http://svn.apache.org/viewvc/hadoop/common/branches/branch-1/ (which I
believe should be running essentially the identical relevant code as
0.20.205
I have tried to run it but it repeats crashing..
- When you start gmetad and Hadoop is not emitting metrics, everything
is peachy.
Right, running just ganglia without running hadoop jobs seems stable for at
least a day..
- When you start Hadoop (and it thus starts emitting metrics
gmetad and Hadoop is not emitting metrics, everything
is peachy.
Right, running just ganglia without running hadoop jobs seems stable for at
least a day..
- When you start Hadoop (and it thus starts emitting metrics), gmetad
cores.
True, with a following error : *** stack
I spent a lot of time to figure it out however i did not find a solution.
Problems from the logs pointed me for some bugs in rrdupdate tool, however
i tried to solve it with different versions of ganglia and rrdtool but the
error is the same. Segmentation fault appears after the following lines
I would be glad to hear that too.. I've setup the following:
Hadoop 0.20.205
Ganglia Front 3.1.7
Ganglia Back *(gmetad)* 3.1.7
RRDTool http://www.rrdtool.org/ 1.4.5. - i had some troubles installing
1.4.4
Ganglia works just in case hadoop is not running, so metrics are not
publshed to gmetad
Hi,
I'm having trouble setting up Hadoop 0.20.2 with Ganglia 3.1. Ganglia is
running, and I am getting standard metrics, but I am not seeing any of the
Hadoop metrics. BTW, I'm running this in EC2.
I applied the GangliaContext31 patch, HADOOP-4675. I believe the patch is
working, as I DO
Hi:
I am using Hadoop 0.20.205. I am getting following error:
Error: java.lang.
ClassNotFoundException: org.slf4j.LoggerFactory
Slf4j API and implementation Jars are in /usr/hadoop/lib folder on namenode as
well as added to classpath via DistributedCache.addArchiveToClasspath . I have
Hi all,
First of all, ganglia integration with hadoop is an awesome feature. Kudos
to the hadoop devs. Unfortunately its not working out for me. I am unable to
see hadoop specific metrics in my ganglia frontend. My configurations are
as follows:
gmetad.coinf :
data_source hadoop test host-name
I downloaded the latest version of Ganglia and compiled and installed
on my Hadoop cluster. Configured according to the documented
procedures. The latest stable version of Ganglia is 3.2, and I am
using hadoop-0.20.2-cdh31
I just copied the gmond.conf from the distribution to the nodes. It
has
Sorry to follow up my own post but I thought I would give it one more
shot this morning and change to dfs.servers=239.2.11.71:8649 (the
multicast address).
Though I am sure I tried that before, it works this time.
Perhaps the Ganglia system was in some unusual state before.
On 09/11/11 08
We have a 2 rack hadoop cluster with ganglia 3.0 monitoring on all stations
both on the native os and within hadoop.
We want to upgrade to the to hadoop 20.203 but with the migration to metrics2
we need help configuring the metrics to continue ganglia monitoring.
All tasktrackers/datanodes
Ganglia plugin is not yet ported to metrics v2 (because Y don't use
Ganglia, see also the discussion links on HADOOP-6728). It shouldn't
be hard to do a port though, as the new sink interface is actually
simpler.
On Wed, May 18, 2011 at 4:07 AM, Eric Berkowitz
eberkow...@roosevelt.edu wrote:
We
Hello,
I have a hadoop cluster with hbase 0.89 and hive 0.70 on ubuntu lucid 64 bit
servers.
I want to use ganglia. I did not install it per apt-get Install because this
gives me ganglia 3.1 but I need ganglia 3.0.x
(http://wiki.apache.org/hadoop/GangliaMetrics).
This didn't help me out
http
You may specify --with-gmetad parameter if you compile ganglia yourself.
For example:
./configure --sysconfdir=/etc/ganglia --with-gmetad
2011/4/11 malte.eh...@gmx.de
Hello,
I have a hadoop cluster with hbase 0.89 and hive 0.70 on ubuntu lucid 64
bit servers.
I want to use ganglia. I did
-an | grep 8649) in this same
config file.
Both should work.
EF
On 26 November 2010 17:15, Mark McCorkle mark.mccor...@medialets.comwrote:
Since I couldn't get my hosted network to work with UDP multicasting, I've
configured my ganglia boxes to talk tcp to one another. Now that I've done
that, I
Since I couldn't get my hosted network to work with UDP multicasting, I've
configured my ganglia boxes to talk tcp to one another. Now that I've done
that, I don't know how to get hadoop to publish data to my ganglia. I'm
running Hadoop 0.20 and gmond 3.07. Below is the important part
Hi,
I have set up ganglia for my cluster, and it works fine. What are
the changes I need to make to make ganglia show hadoop related parameters?
My gmond/gmetad config is default except for one change in gmond.conf for
all nodes:
/* You can specify as many tcp_accept_channels as you like
Hi Hari,
Not sure if you have already followed article -
http://wiki.apache.org/hadoop/GangliaMetrics to configure Ganglia for Hadoop
Monitoring. Also in the details given by you, I do not see any configuration
settings done in gmetad config. You will need to define the hosts from whom
have
done or only the local node? e.g, localhost:8649 everywhere?
I have configured gmetad.conf, and I am able to see metrics from all my
nodes in my ganglia UI. I just don't see any hadoop specific metrics.
Thanks,
Hari
On Mon, Nov 22, 2010 at 6:42 PM, Sagar Shukla sagar_shu...@persistent.co.in
, but just to confirm. (I have set tcp_accept_channel as 8650)
2. In hadoop-metrics.properties, should I mention all my nodes as I have
done or only the local node? e.g, localhost:8649 everywhere?
I have configured gmetad.conf, and I am able to see metrics from all my
nodes in my ganglia UI. I
Hi Matt,
Tried using the specified(default) mcast join ip address. Still no
new metrics. Do I have to restart the hadoop cluster for the changes to take
effect? The new metrics will appear in the Metrics dropdown menu of
ganglia UI home page right? Or is there some other link for custom
) mcast join ip address. Still no
new metrics. Do I have to restart the hadoop cluster for the changes to take
effect? The new metrics will appear in the Metrics dropdown menu of
ganglia UI home page right? Or is there some other link for custom metrics?
thanks,
Hari
On Mon, Nov 22, 2010
PM, Hari Sreekumar wrote:
Hi Matt,
Tried using the specified(default) mcast join ip address. Still no
new metrics. Do I have to restart the hadoop cluster for the changes to
take
effect? The new metrics will appear in the Metrics dropdown menu of
ganglia UI home page right
Hi Brian,
Thank you for the response :)I have used GangliaContext31 in the
properties file and it did not throw any exception when I start up the
cluster. Does this mean that my version is patched? Because I am not able to
see any new metrics in ganglia. What are the additional metrics
Hi,
What are the changes I need to make to run ganglia 3.1.7 on hadoop
0.20.2? I have used GangliaContext31, but I think I'll also need to apply
the 4675 patch https://issues.apache.org/jira/browse/HADOOP-4675. How do I
apply the patch to my distribution? And which of the patched works best
Hi,
I have set up ganglia on my cluster after seeing several
recommendations for using it. I can already see CPU%, free mem, swap space
etc. on it. What additional features should I expect to get after
configuring the hadoop-metrics file? I just want to check if I have
configured
Hi
I have cluster of 4 machines and want to configure ganglia for monitoring
purpose. I have read the wiki and add the following lines to
hadoop-metrics.properties on each machine.
dfs.class=org.apache.hadoop.metrics.ganglia.GangliaContext
dfs.period=10
dfs.servers=10.10.10.2:8649
mapred.class
Hi Shuja,
(First a note: please do not cross-post onto multiple lists, it's considered
rude)
Your configuration looks good, I don't think you'll need to do anything more,
as long as 10.10.10.2:8649 is the correct address. Ganglia can be configured
in many ways, so it's hard for me to tell
Hi Brian,
(I was not sure which is most suitable list for such questions so i post on
multiple lists)
I use telnet and it shows some xml in output and i believe its working. I am
running 3.1.2 version and installed the packages on ubuntu from this link.
http://packages.ubuntu.com/lucid/ganglia
On Nov 8, 2010, at 8:34 AM, Shuja Rehman wrote:
Hi
I have cluster of 4 machines and want to configure ganglia for monitoring
purpose. I have read the wiki and add the following lines to
hadoop-metrics.properties on each machine.
dfs.class=org.apache.hadoop.metrics.ganglia.GangliaContext
Me and a colleague of mine (Ryan Greenhall) setup Ganglia on our hadoop
cluster, he has written a summary of what we did to get it to work, you
might find it useful:
http://forwardtechnology.co.uk/blog/4cc841609f4e6a02114f
Regards,
Abhinay Mehta
On 8 November 2010 15:31, Jonathan Creasy
(Ryan Greenhall) setup Ganglia on our hadoop
cluster, he has written a summary of what we did to get it to work, you
might find it useful:
http://forwardtechnology.co.uk/blog/4cc841609f4e6a02114f
Regards,
Abhinay Mehta
On 8 November 2010 15:31, Jonathan Creasy jon.cre
...@gmail.commailto:abhinay.me...@gmail.comwrote:
Me and a colleague of mine (Ryan Greenhall) setup Ganglia on our hadoop
cluster, he has written a summary of what we did to get it to work, you
might find it useful:
http://forwardtechnology.co.uk/blog/4cc841609f4e6a02114f
Regards,
Abhinay Mehta
On 8
Brian,
Works for me now.. one should point the servers param to the multicast
address that gmond writes to and listens on... and not the ganglia server.
Started working once I did this.
thanks for you inputs,
-G.
On Tue, Aug 24, 2010 at 7:12 PM, Brian Bockelman bbock...@cse.unl.eduwrote
Hi Gautam,
Yup - that's one possible way to configure Ganglia and is common at many sites.
That's why I usually recommend the telnet trick to determine what IP address
your configuration is using.
Brian
On Aug 25, 2010, at 5:53 AM, Gautam wrote:
Brian,
Works for me now.. one should
I was trying to get Ganglia 3.1 to work with the stable hadoop-0.20.2
version from Apache. I patched this release from HADOOP-4675 using
HADOOP-4675-v7.patch as suggested by CDH3 release notes [1] I am unable to
see any hadoop metrics on the Ganglia monitoring UI. The other metrics that
gmond
On Aug 24, 2010, at 8:27 AM, Gautam wrote:
I was trying to get Ganglia 3.1 to work with the stable hadoop-0.20.2
version from Apache. I patched this release from HADOOP-4675 using
HADOOP-4675-v7.patch as suggested by CDH3 release notes [1] I am unable to
see any hadoop metrics
When Hadoop is properly configured to use Ganglia, do you see additional
entries in the Ganglia XML output? We've configured Hadoop to send metrics
to Ganglia, but we're not seeing additional information when we run telnet
localhost 8649. The only configuration change we made was to update
hadoop
Ganglia, do you see additional
entries in the Ganglia XML output? We've configured Hadoop to send metrics
to Ganglia, but we're not seeing additional information when we run telnet
localhost 8649. The only configuration change we made was to update
hadoop-metrics.properties per the instructions
{
mcast_join = 239.2.11.71
port = 8649
ttl = 1
}
And for those who have similar problems getting this to work, here is a
sample of the sort of output to expect from Ganglia if this is working
correctly. A significant number of additional entries will appear in the XML
output...
METRIC NAME
Hi,
I have been trying to get Hadoop working with Ganglia, and am making
some progress.
I have upgraded to Hadoop 0.20.1, and that seems to make a big
difference, I no longer get any errors related to Ganglia.
But when I run gmetad --debug=5, I get the following:
[r...@monitor ganglia
John-
I would recommend that you drop into irc channel #ganglia on freenode or
join the ganglia-general mailing list (http://ganglia.info/) and send this
question there. This seems like a configuration/firewall issue.
-Matt
On Mon, Nov 23, 2009 at 12:41 AM, John Martyniak
j
So if I rollback to Ganglia 3.0.x does this problem go away, and
everything should work?
-John
On Nov 17, 2009, at 9:08 PM, Brian Bockelman wrote:
Hey John,
You need the latest version of this patch for the 0.19.x branch:
http://issues.apache.org/jira/browse/HADOOP-4675
Sadly, the patch
yes
2009/11/18, John Martyniak j...@beforedawnsolutions.com:
So if I rollback to Ganglia 3.0.x does this problem go away, and
everything should work?
-John
On Nov 17, 2009, at 9:08 PM, Brian Bockelman wrote:
Hey John,
You need the latest version of this patch for the 0.19.x branch
Has anybody else had any trouble running hadoop 0.19.2 and Ganglia
3.1.x?
I was surfing through the Jira/Google and it seems that there where
some issues but have been resolved.
Any thoughts would be helpful.
Thank you,
-John
we use cloudera 0.20.1+152 dist. it has included the ganglia 3.1.x patch,
maybe you could try it.
－
天天开心
身体健康
Sent from Nanjing, 32, China
Ted Turner http://www.brainyquote.com/quotes/authors/t/ted_turner.html -
Sports is like a war without the killing.
2009/11/18 John Martyniak j
, 2009, at 8:50 PM, Y G wrote:
we use cloudera 0.20.1+152 dist. it has included the ganglia 3.1.x patch,
maybe you could try it.
－
天天开心
身体健康
Sent from Nanjing, 32, China
Ted Turner http://www.brainyquote.com/quotes/authors/t/ted_turner.html -
Sports is like a war without the killing
(AbstractMe
tricsContext.java:249)
at java.util.TimerThread.mainLoop(Timer.java:534)
at java.util.TimerThread.run(Timer.java:484)
Any ideas?
-John
On Nov 17, 2009, at 8:50 PM, Y G wrote:
we use cloudera 0.20.1+152 dist. it has included the ganglia 3.1.x
patch,
maybe you could try
)
at java.util.TimerThread.mainLoop(Timer.java:534)
at java.util.TimerThread.run(Timer.java:484)
Any ideas?
-John
On Nov 17, 2009, at 8:50 PM, Y G wrote:
we use cloudera 0.20.1+152 dist. it has included the ganglia 3.1.x patch,
maybe you could try it.
－
天天开心
身体健康
Sent from Nanjing, 32
wrote:
we use cloudera 0.20.1+152 dist. it has included the ganglia
3.1.x patch,
maybe you could try it.
－
天天开心
身体健康
Sent from Nanjing, 32, China
Ted Turner http://www.brainyquote.com/quotes/authors/t/ted_turner.html
-
Sports is like a war without the killing.
2009/11/18 John Martyniak j
the
ganglia meta daemon (gmetad) and the front-end PHP scripts. You need
to make sure the firewall rules allow you to make a TCP connection to
port 8649 of your gmond machine.
For more details about installation and configuration, take a look at
our wiki page
http://sourceforge.net/apps/trac
John,
I have been using the scripts provided in the hadoop-ec2-init-remote.sh.
But whenever I try to connect to the master-node for ganglia metrics I
always get the message. server is taking too long to respond. The
documentation says to type in the command
bin/hadoop-ec2 proxy cluster-name
. By
default, ganglia will use multicast to share metrics with other gmond in a
cluster and that won't work on ec2.
For example, let's say that you pick the gmond running on node (e.g.
ip-10-10-10-10.ec2.internal) to listen to every gmond in your cluster. You
will need to update the gmond.conf on ip-10
/GANGLIA_XML
Connection closed by foreign host.
Thanks a lot! . Can I get this information in my local machine browser ?
Thanks and Regards,
Sampritat
On Wed, Sep 9, 2009 at 12:21 AM, Matt Massie m...@cloudera.com wrote:
Ganglia doesn't need to be patched to work. The patches are for Hadoop
Hello All,
I am trying to study the performance of ec2 cluster when it is running
hadoop. But I am not able get ganglia up and running. Can someone please
guide me as how to use/configure Ganglia to be able to run with hadoop . I
am using an public ec2 image that has hadoop-0.19.0. I have used
I should be able to help you out.
What AMI are you using? What linux distribution?
-Matt
On Tue, Sep 8, 2009 at 6:21 PM, Samprita Hegde samprit...@gmail.com wrote:
Hello All,
I am trying to study the performance of ec2 cluster when it is running
hadoop. But I am not able get ganglia up
The ami-id I am using is ami-fa6a8e93. It is a Fedora distribution for i-386
.
To be precise this is taken from the console output :
Linux version 2.6.21.7-2.fc8xen (mockbu...@xenbuilder1.fedora.redhat.com)
(gcc version 4.1.2 20070925 (Red Hat 4.1.2-33))
I was looking into the Ganglia
73 matches
Mail list logo
