[Ganglia-general] gmond forwarding from one internal network to another?
Hi All,
I think this topic was discussed here; but somehow I am stuck with it
again.
I have a unicast gmond setup on our cluster. There are compute nodes on
192.168, login nodes on 10.x and 192.168 and the Ganglia server on 10.x
only. I want to forward metrics from 192.168 to the server, preferably
without creating custom routing schemes as it might affect other stuff on
the cluster. So I have following setup:
1) Compute node's gmond are deaf and send metrics over 192.168 network
to three of login nodes gmond's. So I have three send channels like this:
udp_send_channel {
bind_hostname = yes
host = 192.168.x1.y1 port = 8649
ttl = 3
}
2) Login nodes receive and send metrics. I want them to receive computes'
metrics over their 192.168 interface and pass metrics to 10.0. network,
where my gmond/gmetad/gweb resides. So their send channel is:
udp_send_channel {
bind_hostname = yes
host = 10.my-gmetad-ip
port = 8649 ttl = 2
}
and they supposed to listen to anything?
udp_recv_channel {
port = 8649
# Size of the UDP buffer. If you are handling lots of metrics you really
# should bump it up to e.g. 10MB or even higher.
buffer = 20971520
}
3) and the gmond of 10.my-gmetad-ip has similar udp_recv_channel config.
Somehow it doesn't work though; telnet to the TCP port on the server,
10.my-gmetad-ip shows that it gives only the metrics from the login nodes
coming from their 10.x interfaces. The login nodes seem to have the
metrics from compute nodes' 192.168.x interface, as telnet shows, but do
not pass these further, somehow.
I check the metrics as follows:
telnet 10.0.0.24 8649 | grep HOST
Could you please suggest, what am I doing wrong in this setup? Is gmond
supposed to forward like this at all? Thank you very much in advance!
--
Grigory Shamov
Westgrid/ComputeCanada Site Lead
University of Manitoba
E2-588 EITC Building,
(204) 474-9625
--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151=/4140
___
Ganglia-general mailing list
Ganglia-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ganglia-general
Re: [Ganglia-general] gmond forwarding
On 10/27/2011 08:36 PM, Rick Cobb wrote: In your case, your sender doesn't know about the other receivers and will need to be configured via its own configuration technique. It can be confusing that gmond is a daemon with three roles (and gmetad one with two), but that's the way it is. Thanks. I think I understand gmond's multiple rules much better, and was able to get my setup working. -- RSAreg; Conference 2012 Save #36;700 by Nov 18 Register now http://p.sf.net/sfu/rsa-sfdev2dev1 ___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general
[Ganglia-general] gmond forwarding
I have a unicast setup where each cluster box is sending to 3 collector servers. For all sorts of normal metrics (cpu load) etc this works fine. Each cluster box also has several java applications [1] are also sending metrics to the gmond on localhost. Running with debug=2, I see that those metrics are received by gmond, but they are not send over the udp_send_channel to the primary collector gmonds. I don't see any knobs in gmond.conf, so I am confused how gmond would get a metrics, but not forward it. In general, does gmond always forward metrics over it's send channel? [1] Using https://github.com/codahale/metrics/tree/development/metrics-ganglia This is similar to the ganglia code in Hadoop and jmxtrans. -- The demand for IT networking professionals continues to grow, and the demand for specialized networking skills is growing even more rapidly. Take a complimentary Learning@Cisco Self-Assessment and learn about Cisco certifications, training, and career opportunities. http://p.sf.net/sfu/cisco-dev2dev ___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general
Re: [Ganglia-general] gmond forwarding
No, it does not. Vladimir wrote up a nice tutorial on the wiki, but I don't remember whether he covered the Java library for sending Ganglia metrics. The way I think of the Ganglia architecture is as: * There are measurement *senders*. Gmond is one of them, but the Java library for sending metrics is a different one, as is gmetric. Every sender has to be told where to send its measurements. gmetric is nice enough to read /etc/ganglia/gmond.conf, but the Java library definitely *isn't*; it has its own properties area. I don't know whether that's been extended over the years to handle multiple destinations; I presume it has been since people run it in clouds that don't support multicast. * There are measurement *receivers*. gmond is the only one of these in the Ganglia architecture. These are the daemons that get measurements. * There are measurement *aggregators*. An aggregator gets TCP requests for whole boatloads of measurements, and returns them as XML. gmond is the main one of these in Ganglia. OTOH, gmetad is also an aggregator, and is used directly as one when you're doing grids of grids. * There are measurement *recorders*. Gmetad is the only one of htese in the Ganglia architecture; it records measurements for posterity in RRDtool files. * There are measurement-history presentation tools. gweb does this. It uses gmetad to get the current aggregated grid-level measurements, and the recorded RRDtool information to show the historical context. In your case, your sender doesn't know about the other receivers and will need to be configured via its own configuration technique. It can be confusing that gmond is a daemon with three roles (and gmetad one with two), but that's the way it is. Hope this helps -- -- ReC On Thu, Oct 27, 2011 at 11:50 AM, Chris Burroughs chris.burrou...@gmail.com wrote: I have a unicast setup where each cluster box is sending to 3 collector servers. For all sorts of normal metrics (cpu load) etc this works fine. Each cluster box also has several java applications [1] are also sending metrics to the gmond on localhost. Running with debug=2, I see that those metrics are received by gmond, but they are not send over the udp_send_channel to the primary collector gmonds. I don't see any knobs in gmond.conf, so I am confused how gmond would get a metrics, but not forward it. In general, does gmond always forward metrics over it's send channel? [1] Using https://github.com/codahale/metrics/tree/development/metrics-ganglia This is similar to the ganglia code in Hadoop and jmxtrans. -- The demand for IT networking professionals continues to grow, and the demand for specialized networking skills is growing even more rapidly. Take a complimentary Learning@Cisco Self-Assessment and learn about Cisco certifications, training, and career opportunities. http://p.sf.net/sfu/cisco-dev2dev ___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general -- The demand for IT networking professionals continues to grow, and the demand for specialized networking skills is growing even more rapidly. Take a complimentary Learning@Cisco Self-Assessment and learn about Cisco certifications, training, and career opportunities. http://p.sf.net/sfu/cisco-dev2dev___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general
