[Ganglia-general] Best topology for unicast deployment
Hi, I'd like to know what's the best topology for a unicast deployment. I want to monitor several clusters (memcache / redis / mysql / etc...). --- One solution is to configure each node in a given cluster to talk to a given head gmond (or two for HA) M1, M2, M3 -> send to MHEAD:MPORT R1, R2, R3 -> send to RHEAD:RPORT ... a gmetad is configured to get data from all the head gmond MHEAD:MPORT, RHEAD:RPORT, etc datasource 'memcache' MHEAD:MPORT datasource 'redis' RHEAD:RPORT since all my head gmond and my gmetad are on a single box, this is a lot of gmond processes listening to different ports (handled by supervisor) --- The other solution is to configure each node in a given cluster to talk to each other: M1 -> sends to M2,M3 M2 -> sends to M1,M3 M3 -> sends to M1,M2 R1 -> sends to R2,R3 R2 -> sends to R1,R3 R3 -> sends to R1,R2 ... And have gmetad talk to a given machine for each cluster (adding all the others for HA) datasource 'memcache' M1, M2, M3 datasource 'redis R1, R2, R3 This is more chatty. Are both solutions acceptable or is there some sort of consensus on what to use? Also I have data send by gmetric and some time ago it seemed that the data where send to the send_channel but not localhost. This could be a problem in the second scenario where we don't get the data for the first host. Is that an expected behavior of gmetric, a bug or did I do something wrong? Thanks --Gilles -- Try before you buy = See our experts in action! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-dev2 ___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general
Re: [Ganglia-general] Ganglia gmond memory leak?
Hi Aidan, for what it is worth, I cannot reproduce the growing memory consumption on a small 3.2.0 grid using only standard metrics in unicast mode. Running now for a few hours. Will check again tomorrow. Cheers Martin -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de > > From: Aidan Wong >To: "Ave-Lallemant, Nathan P" ; >ganglia-general >Sent: Thursday, February 23, 2012 8:34 AM >Subject: Re: [Ganglia-general] Ganglia gmond memory leak? > > >I've restarted the gmond process and memory usage drops until gmond hogs >memory over time. Any Ganglia contributors who may want to chime in on this >memory leak issue? I'm on Ganglia 3.2.0. Are there any improvements on >version 3.3.1 addressing this issue? > > >Thanks > >From: "Ave-Lallemant, Nathan P" >Date: Wed, 22 Feb 2012 16:31:58 -0600 >To: Aidan Wong , ganglia-general > >Subject: RE: Ganglia gmond memory leak? > > > > >I have seen the same behavior in my environment but do not have a solution. > > >Nathan > > > >From:Aidan Wong [mailto:aidanw...@attinteractive.com] >Sent: Wednesday, February 22, 2012 4:10 PM >To: ganglia-general >Subject: [Ganglia-general] Ganglia gmond memory leak? > >Hi it looks like my install of gmond version 3.2.0 is leaking memory. The >amount of resident used memory that the process uses, gets up pretty high and >keeps increasing. > >USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND >root 18647 0.0 9.9 2965464 1836268 ? Ss Jan14 11:24 >/home/t/hadoop-ganglia-client/sbin/gmond -c >/home/t/hadoop-ganglia-client/gmond.conf -p >/home/t/hadoop-ganglia-client/logs/gmond.pid > >Is this a bug? Can anyone suggest a solution? > >Thank you >> > CONFIDENTIALITY NOTICE: This e-mail and any files transmitted with it are > intended solely for the use of the individual or entity to whom they are > addressed and may contain confidential and privileged information protected > by law. If you received this e-mail in error, any review, use, dissemination, > distribution, or copying of the e-mail is strictly prohibited. Please notify > the sender immediately by return e-mail and delete all copies from your > system. > > > >-- >Virtualization & Cloud Management Using Capacity Planning >Cloud computing makes use of virtualization - but cloud computing >also focuses on allowing computing to be delivered as a service. >http://www.accelacomm.com/jaw/sfnl/114/51521223/ >___ >Ganglia-general mailing list >Ganglia-general@lists.sourceforge.net >https://lists.sourceforge.net/lists/listinfo/ganglia-general > > >-- Try before you buy = See our experts in action! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-dev2___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general
[Ganglia-general] tcpconn.py and netstat
Currently tcpconn.py uses netstat to get it's socket stats. This gives lots of detail but is far too slow for much production use (running netstat can take many minutes). /proc/net/sockstat gives less information but has no performance problems. There was a suggestion previously to use the ss command, but (1) it's less common (at least not part of the default on RHEL5) and (2) it also lacks the high fidelity details. Is there any other reason to prefer ss over cat? Should this replace tcpconn, or be a new module? Example output: $ cat /proc/net/sockstat sockets: used 292 TCP: inuse 1103 orphan 974 tw 701725 alloc 1104 mem 1096 UDP: inuse 6 mem 1 UDPLITE: inuse 0 RAW: inuse 0 FRAG: inuse 0 memory 0 -- Try before you buy = See our experts in action! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-dev2 ___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general
Re: [Ganglia-general] Ganglia gmond memory leak?
I've also observed this and have been unable to find a solution. In my case at least there was no obvious correlation with the number of metrics or weather the gmond was an aggregating or not (so several orders of magnitude in the number of metrics did not matter, it might happen on 2 out of 80 nodes). gmond would take up memory > physical RAM, swap, and general sadness. I'm unfortunately not able to provide further information since we went to nightly gmond restarts as a work around. On 02/22/2012 05:10 PM, Aidan Wong wrote: > Hi it looks like my install of gmond version 3.2.0 is leaking memory. The > amount of resident used memory that the process uses, gets up pretty high and > keeps increasing. > > USER PID %CPU %MEMVSZ RSS TTY STAT START TIME COMMAND > root 18647 0.0 9.9 2965464 1836268 ? Ss Jan14 11:24 > /home/t/hadoop-ganglia-client/sbin/gmond -c > /home/t/hadoop-ganglia-client/gmond.conf -p > /home/t/hadoop-ganglia-client/logs/gmond.pid > > Is this a bug? Can anyone suggest a solution? > > Thank you > > > > > -- > Virtualization & Cloud Management Using Capacity Planning > Cloud computing makes use of virtualization - but cloud computing > also focuses on allowing computing to be delivered as a service. > http://www.accelacomm.com/jaw/sfnl/114/51521223/ > > > > ___ > Ganglia-general mailing list > Ganglia-general@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/ganglia-general -- Try before you buy = See our experts in action! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-dev2 ___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general
[Ganglia-general] List memory used per process
Hi Is there an easy way of getting Ganglia to monitor memory use per process. For example to get the 10 most memory hungry procesess? Regards Peter Ellevseth -- Try before you buy = See our experts in action! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-dev2___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general
Re: [Ganglia-general] Ganglia Web interface - Where to specify an arbitrary host name and IP to be shown in the UI.
I would appreciate if you could share the hardware specification needs to be there for Ganglia. 1. Server running gmond (around 50) 2. Since I am using unicast mode what should be the hardware configuration for the master server(chosen from one of 50 gmond instances) which is gathering data from all gmond running servers in a cluster. 3. One more thing I want to use gmetad along with Ganglia Web Interface in a separate server and the data-source would be the master node from unicast mode(running around 50 gmond). What should be the hardware configure for above mentioned three scenarios. Please do let me know as cost matters in Cloud environment. Thanks, Mozammil From: Mohd Mozammil khan To: Michael Bravo Cc: "ganglia-general@lists.sourceforge.net" Sent: Monday, 27 February 2012 5:38 PM Subject: Re: [Ganglia-general] Ganglia Web interface - Where to specify an arbitrary host name and IP to be shown in the UI. Yes you are right there is a little dance among host-names to be despaired from UI and many times master node in unicast mode takes time to fetch all the required data like - cpu_num, etc. By the way I was missing something to use with override_hostname is, override_ip. Without this gmond assumes IP to be override_hostname. That is really funny :) Well, thanks again Michael. Hope will have good journey ahead with Ganglia. Cheers Mozammil From: Michael Bravo To: Mohd Mozammil khan Cc: "ganglia-general@lists.sourceforge.net" Sent: Monday, 27 February 2012 5:13 PM Subject: Re: [Ganglia-general] Ganglia Web interface - Where to specify an arbitrary host name and IP to be shown in the UI. Greetings, I think you need the override_hostname statement in your gmond configuration. It is available starting with Ganglia 3.2.0 and is described here - http://sourceforge.net/apps/trac/ganglia/wiki/override_hostname Of course, you will need to restart the gmond daemon after changing configuration, and there's always a little dance of restarting gmond/gmetad combination to make the "old" hostnames go away :) Hope this helps, -- Michael Bravo On Mon, Feb 27, 2012 at 1:37 PM, Mohd Mozammil khan wrote: > Hi, > > Greeting !!! > > Have installed Ganglia in our Amazon EC2 environment using unicast mode and > things are working smoothly. There is a little concern about changing the > complex Node name assigned by Amazon EC2 environment some user-friendly name > like - (domU-29-68-59-89-2B-85.compute-1.internal to dev1.domain.com). > After digging a lot, got to know that there is a feature to change > the arbitrary host name and IP to be shown in the UI. > > Request you guys to please help me out of this issue and guide me which file > needs to be edited and what would be the syntax in Ganglia Web Interface to > change a hostname to be shown on Web Interface. Any immediate response would > be highly appreciated. > > > > Cheers, > Mohd Mozammil Khan -- Try before you buy = See our experts in action! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-dev2 ___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general-- Try before you buy = See our experts in action! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-dev2___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general
Re: [Ganglia-general] Ganglia Web interface - Where to specify an arbitrary host name and IP to be shown in the UI.
Yes you are right there is a little dance among host-names to be despaired from UI and many times master node in unicast mode takes time to fetch all the required data like - cpu_num, etc. By the way I was missing something to use with override_hostname is, override_ip. Without this gmond assumes IP to be override_hostname. That is really funny :) Well, thanks again Michael. Hope will have good journey ahead with Ganglia. Cheers Mozammil From: Michael Bravo To: Mohd Mozammil khan Cc: "ganglia-general@lists.sourceforge.net" Sent: Monday, 27 February 2012 5:13 PM Subject: Re: [Ganglia-general] Ganglia Web interface - Where to specify an arbitrary host name and IP to be shown in the UI. Greetings, I think you need the override_hostname statement in your gmond configuration. It is available starting with Ganglia 3.2.0 and is described here - http://sourceforge.net/apps/trac/ganglia/wiki/override_hostname Of course, you will need to restart the gmond daemon after changing configuration, and there's always a little dance of restarting gmond/gmetad combination to make the "old" hostnames go away :) Hope this helps, -- Michael Bravo On Mon, Feb 27, 2012 at 1:37 PM, Mohd Mozammil khan wrote: > Hi, > > Greeting !!! > > Have installed Ganglia in our Amazon EC2 environment using unicast mode and > things are working smoothly. There is a little concern about changing the > complex Node name assigned by Amazon EC2 environment some user-friendly name > like - (domU-29-68-59-89-2B-85.compute-1.internal to dev1.domain.com). > After digging a lot, got to know that there is a feature to change > the arbitrary host name and IP to be shown in the UI. > > Request you guys to please help me out of this issue and guide me which file > needs to be edited and what would be the syntax in Ganglia Web Interface to > change a hostname to be shown on Web Interface. Any immediate response would > be highly appreciated. > > > > Cheers, > Mohd Mozammil Khan-- Try before you buy = See our experts in action! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-dev2___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general
Re: [Ganglia-general] Ganglia Web interface - Where to specify an arbitrary host name and IP to be shown in the UI.
Greetings, I think you need the override_hostname statement in your gmond configuration. It is available starting with Ganglia 3.2.0 and is described here - http://sourceforge.net/apps/trac/ganglia/wiki/override_hostname Of course, you will need to restart the gmond daemon after changing configuration, and there's always a little dance of restarting gmond/gmetad combination to make the "old" hostnames go away :) Hope this helps, -- Michael Bravo On Mon, Feb 27, 2012 at 1:37 PM, Mohd Mozammil khan wrote: > Hi, > > Greeting !!! > > Have installed Ganglia in our Amazon EC2 environment using unicast mode and > things are working smoothly. There is a little concern about changing the > complex Node name assigned by Amazon EC2 environment some user-friendly name > like - (domU-29-68-59-89-2B-85.compute-1.internal to dev1.domain.com). > After digging a lot, got to know that there is a feature to change > the arbitrary host name and IP to be shown in the UI. > > Request you guys to please help me out of this issue and guide me which file > needs to be edited and what would be the syntax in Ganglia Web Interface to > change a hostname to be shown on Web Interface. Any immediate response would > be highly appreciated. > > > > Cheers, > Mohd Mozammil Khan -- Try before you buy = See our experts in action! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-dev2 ___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general
[Ganglia-general] Ganglia Web interface - Where to specify an arbitrary host name and IP to be shown in the UI.
Hi, Greeting !!! Have installed Ganglia in our Amazon EC2 environment using unicast mode and things are working smoothly. There is a little concern about changing the complex Node name assigned by Amazon EC2 environment some user-friendly name like - (domU-29-68-59-89-2B-85.compute-1.internal to dev1.domain.com). After digging a lot, got to know that there is a feature to change the arbitrary host name and IP to be shown in the UI. Request you guys to please help me out of this issue and guide me which file needs to be edited and what would be the syntax in Ganglia Web Interface to change a hostname to be shown on Web Interface. Any immediate response would be highly appreciated. Cheers, Mohd Mozammil Khan -- Try before you buy = See our experts in action! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-dev2___ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general
