> -----Original Message----- > From: ganglia-general-requ...@lists.sourceforge.net [mailto:ganglia- > general-requ...@lists.sourceforge.net] > Sent: 24 October 2012 00:35 > To: ganglia-general@lists.sourceforge.net > Subject: Ganglia-general Digest, Vol 77, Issue 34 > > Send Ganglia-general mailing list submissions to > ganglia-general@lists.sourceforge.net > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.sourceforge.net/lists/listinfo/ganglia-general > or, via email, send a message with subject or body 'help' to > ganglia-general-requ...@lists.sourceforge.net > > You can reach the person managing the list at > ganglia-general-ow...@lists.sourceforge.net > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Ganglia-general digest..." > > > Today's Topics: > > 1. Question about scaling (Potter,Mark L) > 2. Re: Question about scaling (Nicholas Satterly) > 3. Re: Question about scaling (Potter,Mark L) > 4. Re: Adding Holt-Winters databases to existing rrd causes > __SummaryInfo__ metric to fail to render on graphs (Aaron Nichols) > 5. Re: Question about scaling (Nicholas Satterly) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 23 Oct 2012 12:58:02 -0500 > From: "Potter,Mark L" <mlpot...@mdanderson.org> > Subject: [Ganglia-general] Question about scaling > To: "ganglia-general@lists.sourceforge.net" > <ganglia-general@lists.sourceforge.net> > Message-ID: > <622D99D1851E994CBEB3A7A0C54123722FBFCC8A1A@DCPWVMBXC1VS3.mdanders > on.edu> > > Content-Type: text/plain; charset="us-ascii" > > I am using what I think to be a fairly standard gmond.conf: > > globals { > daemonize = yes > setuid = yes > user = nobody > debug_level = 0 > max_udp_msg_len = 1472 > mute = no > deaf = no > allow_extra_data = yes > host_dmax = 86400 /*secs. Expires (removes from web interface) hosts > in 1 day */ > host_tmax = 30 /*secs */ > cleanup_threshold = 300 /*secs */ > gexec = no > send_metadata_interval = 30 /*secs */ > } > > cluster { > name = "MDACC" > owner = "MD Anderson Caner Center" > latlong = "unspecified" > url = "unspecified" > } > > host { > location = "8,3,1" > } > > udp_send_channel { > host = cnode340 > port = 8649 > } > > udp_recv_channel { > port = 8649 > retry_bind = true > } > > tcp_accept_channel { > port = 8649 > } > > gmetad is set to check every 60 seconds: > > data_source "MDACC" 60 cnode340:8649 > > > Everything works well until around 200 hosts where it appears gmetad > starts having issues. I have ~340 hosts to go in to this cluster. Should > I be running multiple gmetads for this amount of hosts? With all of them > active the web interface reports all of them down and collects no stats > at all. I am looking for advice on getting this up and running properly. > The ganglia host isn't underpowered at all IMO and has plenty of HDD > space: > > Mem: 32955788 (from free) > 16 Cores (AMD Opteron(tm) Processor 6128) > > Thanks for any assistance. > > > Respectfully, > > Mark L. Potter > Research IS & Technology Services > UNIX Systems Administrator > O: 713-745-2032 > C: 713-965-4133 > >
Hi Mark I had a similar problem and solved it by increasing the udp kernel buffers. In /etc/sysctl.conf: # net.core.rmem_max=20480000 net.core.rmem_default=10240000 and specify buffer size in /etc/ganglia/gmond.conf: udp_recv_channel { port = 8649 buffer = 20000000 } If you want to see if you are losing udp packets type: watch -d grep '^Udp' /proc/net/snmp Udp: InDatagrams NoPorts InErrors OutDatagrams Udp: 39230574570 5121159 88139646 608369019 and see if the InErrors field is increasing. Regards -- Paul Hewlett "Write documentation as if whoever reads it is a violent psychopath who knows where you live." Steve English, as quoted by Peter Langston http://www.quotegarden.com/programming.html ARM Ltd 110 Fulbourn Road, Cambridge, CB1 9NJ Tel: +44 (0)1223 405923 skype: paul-at-arm www.arm.com -- IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. ------------------------------------------------------------------------------ Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct _______________________________________________ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general