Rick Cobb wrote: > Changed subjects because this part of the discussion is important > enough to have its own thread > > On Dec 20, 2009, at 8:55 AM, Jesse Becker wrote: > >> On Sun, Dec 20, 2009 at 11:02, Spike Spiegel <fsm...@gmail.com >> <mailto:fsm...@gmail.com>> wrote: >> ... >>> I think there's a middle ground here that'd be interesting to explore, >>> altho that's a different thread, but for kicks this is the gist: the >>> common pattern for rrd storage is hour/day/month/year and I've always >>> found it bogus. In many cases I've needed higher resolution (down to >>> the second) for the last 5-20 minutes, then intervals of an hr to a >>> couple hrs, then a day to three days and then a week to 3 weeks etc >>> etc, which increases your storage requirements, but is imho not an >>> abuse of rrd and still retains the many advantages of rrd over having >>> to maintain a RDBMs. >> >> The d/w/m/y split is a good *starting point*. Ganglia needs to ship >> with some sort of sensible default configuration that essentially >> works for many/most people. You (singular or plural) are are free to >> customize your RRD configuration as policy and storage capacity >> require and permit. Ganglia officially supports this via the RRD >> config like in gmetad.conf. and as your storage system permits. In >> the ideal world, you keep all data, at the highest resolution, >> forever, but that usually isn't practical. >> > > We like to have different resolutions for specific metrics, which > Ganglia doesn't support directly in any way. Also, as far as scaling > the data "up" goes, it would be very handy to retain high-resolution > longer for just the summary metrics (they're what you need for > planning anyway). > ...
> At the moment, this is pretty hard to set up -- you have to create > your rrdfile outside Ganglia with the specialized policy you want. > When thinking about how to provide plugins or other extensibility > hooks for gmetad, this is among the first places I'd look: providing > the rrd create helper function with all of the meta-data about a > metric (is it cluster or grid summary? is it host? hostname? metric > name? Other annotations (we're 3.0, so all our annotations are in the > metric naming convention)?) and making the configuration language > flexible enough to get a whole little pattern language into how to > decide what RRAs to have. This is a potential requirement for me too. One way around it is to run multiple instances of gmetad, each with a different config file. This works if you want to apply a standard RRA to all the metrics from a particular cluster. If you want to do stuff within the node level (e.g. keep CPU data for 24 hours, but keep memory data for 7 days), then it can't be done with the current implementation of gmetad. However, it isn't so hard to extend gmetad to do this. My solution would be an XML based configuration for gmetad. You could then declare multiple RRA templates, and use some kind of regex or heirarchy to apply them to different metrics. ------------------------------------------------------------------------------ This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev _______________________________________________ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers
