On Mon, 2013-01-14 at 00:05 -0800, David Lang wrote: > On Mon, 14 Jan 2013, Conrad Wood wrote: > > > I am looking for a solution to gather, correlate and graph large sets of > > data. > > > > The data is gathered from several hundred servers (going upto multi-1000 > > within 12 month). > > Requirements are: > > * 1 minute resolution of data > > * individual graphs of a single value stream (e.g. IOPS for a given > > block device) > > * Graphs over a bunch of value streams (e.g. max IOPS per minute on > > 100ish block devices together) > > * Data Retention in this resolution for 90 days > > > > I looked at: > > Collectd - missing data correlation and probably cannot handle large > > amounts of data > > > > Graphite/Carbon - missing data correlation > > > > RRD - it averages data, which is *NOT* what I want. I need the absolute > > MAX values in 1-Minute intervals > > RRD can be configured to keep data is whatever resolution you want for > whatever > time period you want. By default is averages things, but I've configured RRD > databases to have 1 min intervals and to keep that resoution for longer time > periods. > > Whenyou create a RRD datastore, you tell it what time periods to use, and how > long to keep them for each time period. You also tell is if you want to store > MAX as well as average values for that series of data points.
Thanks for the swift response. If I calculate 300 Servers, each providing data of approx 30 64bit values in 1 minute intervals I get 9000 Values per minute. That is 12,960,000 values per day. so for 90 days it is 1,166,400,000 values, I need to store. I don't imagine RRD being particular efficient at that. Further more with RRD datastore I stick one stream of values in one database and have no way of correlating them. Or am I mistaken here? Conrad _______________________________________________ Tech mailing list [email protected] https://lists.lopsa.org/cgi-bin/mailman/listinfo/tech This list provided by the League of Professional System Administrators http://lopsa.org/
