Re: [Ganglia-developers] Gmetad bottlenecks

2013-12-07 Thread Devon H. O'Dell
2013/12/7 Chris Burroughs : > Thank you Devon and Vladimir for starting this thread. We (AddThis) > have been struggling with gmetad performance and stability for a while > and I'm personally excited to see the focus here. I'll explain briefly > how we are using ganglia for context and then have

Re: [Ganglia-developers] Gmetad bottlenecks

2013-12-07 Thread Nikhil
Thank you Vladimir and Devon. Much appreciated. +2 for these below initiatives. >> * changing the data serialization format from XML to one that is easier / faster to parse, * using a different data structure than a hash table for metrics hierarchies (probably a tree with metrics stored at each

Re: [Ganglia-developers] Gmetad bottlenecks

2013-12-07 Thread Vladimir Vuksan
On 12/07/2013 02:23 PM, Chris Burroughs wrote: > On 12/06/2013 03:36 PM, Vladimir Vuksan wrote: >> The Ganglia core is comprised of two daemons, `gmond` and `gmetad`. >> `Gmond` is >> primarily responsible for sending and receiving metrics; `gmetad` >> carries the >> hefty task of summarizing / a

Re: [Ganglia-developers] Gmetad bottlenecks

2013-12-07 Thread Chris Burroughs
Thank you Devon and Vladimir for starting this thread. We (AddThis) have been struggling with gmetad performance and stability for a while and I'm personally excited to see the focus here. I'll explain briefly how we are using ganglia for context and then have inline comments. We have two dat

Re: [Ganglia-developers] Gmetad bottlenecks

2013-12-07 Thread Peter Phaal
Adrian, Have you experimented with Host sFlow agents? http://host-sflow.sourceforge.net/ The Host sFlow agents export the following standard metrics (which are based on the Ganglia libmetrics core, adding additional metrics for disk I/O, UUIDs, virtual machines etc.): http://sflow.org/sflow_h

Re: [Ganglia-developers] Gmetad bottlenecks

2013-12-07 Thread Devon H. O'Dell
2013/12/7 Adrian Sevcenco : > On 12/06/2013 10:51 PM, Devon H. O'Dell wrote: >> 2013/12/6 Vladimir Vuksan : >>> Hello everyone, > Hi! > >>> For few weeks now we have had performance issues due to growth of >>> our monitoring setup. One of my colleagues Devon O'Dell volunteered >>> to help and below

Re: [Ganglia-developers] Gmetad bottlenecks

2013-12-07 Thread Devon H. O'Dell
2013/12/6 Nicholas Satterly : > Excellent work, Devon. A few comments... > > What happens when the RRD TCP receive queue fills up and the socket blocks > or if the TCP socket is closed by RRDcached for some reason? Does each > gmetad data thread block/time out/retry the connection? On connection e