Hi Daniel: On Fri, Jul 4, 2008 at 1:52 AM, <[EMAIL PROTECTED]> wrote:
> How scalable is gmetad meant to be? How many nodes and/or metrics based > on the assumption that samples are to be taken very frequently, > typically less than 10 seconds apart? > > Has anyone reached any limitations in terms of network throughput for > gathering data, or IO throughput for writing to the RRD files? > > Currently, it appears that scalability is achieved by dividing a server > farm into groups of servers, and linking each group to a dedicated > gmetad server. However, is there a way to operate gmetad servers in a > pool, possibly using shared storage for RRD files? > > This is one area that I might be able to contribute to, as I suspect our > server estate is bigger than what a single gmetad can handle, especially > if we add additional metrics. It would be useful to know what existing > work has been done on these issues. Currently there are scalability issues when you monitor thousands of nodes with one gmetad. The current implementation of Ganglia store each hosts' metrics as separate rrd files so as you increase the number of hosts, the number of total rrd files increment by 30+ (by default we collect roughly 30+ metrics for each host). The current "workaround" for this scalability issue is to store the rrds in tmpfs, which seems to have alleviated the problem for most people. There are currently a lot of discussions in the rrd mailing-lists regarding ways to "accelerate" rrdtool (and Ganglia relies on rrdtool). I encourage you to read up on those discussions and potentially participate: http://www.mail-archive.com/[EMAIL PROTECTED]/msg02238.html http://www.mail-archive.com/[EMAIL PROTECTED]/msg14362.html Regarding distributed rrdtool, I believe that is also something the rrd developers are working on, and I believe once that is implemented, we will be able to provide this feature to Ganglia users. Thanks for your interest. Cheers, Bernard ------------------------------------------------------------------------- Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08 _______________________________________________ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers
