On Sun, Dec 20, 2009 at 04:02:36PM +0000, Spike Spiegel wrote: > On Mon, Dec 14, 2009 at 10:28 AM, Carlo Marcelo Arenas Belon > <care...@sajinet.com.pe> wrote: > > > >> b) you can afford to have duplicate storage - if your storage > >> requirements are huge (retaining a lot of historic data or lot's of data > >> at short polling intervals), you may not want to duplicate everything > > > > if you are planning to store a lot of historic data then you should be > > using instead some sort of database, not RRDs and so I think this shouldn't > > be an issue unless you explode the RRAs and try to abuse the RRDs as a RDBMs > > I think there's a middle ground here that'd be interesting to explore, > altho that's a different thread, but for kicks this is the gist: the > common pattern for rrd storage is hour/day/month/year and I've always > found it bogus.
I am sure the defaults provided were completely arbitrary (I think you missed week) but make sense based on the fact that there were the smallest time unit of their kind and also that they fit the standard gmond polling rates which wouldn't accommodate for 1 min or 1 sec. > In many cases I've needed higher resolution (down to > the second) for the last 5-20 minutes, then intervals of an hr to a > couple hrs, then a day to three days and then a week to 3 weeks etc > etc, which increases your storage requirements, but is imho not an > abuse of rrd and still retains the many advantages of rrd over having > to maintain a RDBMs. agree, and the fact that it is not easy enough to do or requires a somehow intrusive maintenance is a bug, but still possible for the reasons you explain. > > PS. I like the ideas on this thread, don't get me wrong, just that I agree > > ?? ??with Vladimir that gmetad and RRDtool are probably not the sweet spot > > ?? ??(cost wise) for scalability work even if I also agree that the vertical > > ?? ??scalability of gmetad is suboptimal to say the least. > > sort of. If you're looking at where your resources go to compute and > deal with large amount of data, I agree. If you look at what it costs > you or if it's even possible to create a fully scalable and resilient > ganglia based monitoring infrastructure, I disagree. not sure what part are you quoting here, but I have the feeling we probably agree ;) getting my ganglia developer hat, I dislike the fact that gmetad can't scale horizontally like all well designed applications should, but the fact that there is no solution for it to do so yet, means that the complexity involved on making that change is probably not worth it in most (if not all) the cases considering that hardware (to the levels needed most of the time) is cheap anyway, as I really hope there is no one out there running gmetad in some big iron solution, when some decent "PC" box with enough memory would do mostly fine. there are problems as well with the way federation currently works which require more network bandwith and CPU that should be really needed and that I would guess we should tackle first, specially considering the increase of the XML sizes with 3.1 (which also has been worked around too) but for that (getting my ganglia user hat) would assume most big installations will stick with 3.0 anyway for now. Carlo ------------------------------------------------------------------------------ This SF.Net email is sponsored by the Verizon Developer Community Take advantage of Verizon's best-in-class app development support A streamlined, 14 day to market process makes app distribution fast and easy Join now and get one step closer to millions of Verizon customers http://p.sf.net/sfu/verizon-dev2dev _______________________________________________ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers
