On Tue, Nov 13, 2007 at 09:28:13AM -0800, Jason Fesler wrote:
> Nope. SEnd the oldest data first. Maybe don't send it all at once - but
> record into rrd from your queue oldest first.
what we do is to drop one entry out of two, so we don't get any gaps (as
long as our heartbeat=4*step isn't at
Mike,
One workaround I've seen for the huge volume of metric data is to
break your large RRD files with multiple DS's into multiple RRD files
with a single (or small number of) DS. If you must avoid a spike in
your traffic, catching up with all the data at once impossible (i.e.,
you must throw so
> Well, since we're a commercial service, we don't have as much
> flexibility in our data retention policy. I'll discuss with the team
> here - I certainly don't want to muck with RRDtool's innards.
If you're too far behind, you might consider a ramdisk. Copy .rrd to
ramdisk, then do all the up
Mike Perham wrote:
> We have a monitoring service which collects metrics. If that service
> is down, we queue up the stats to be pushed into RRD later. Once the
> service comes back online, we want to push the latest metric data
> along with the older data (not all at once so as to not overwhelm
Well, since we're a commercial service, we don't have as much
flexibility in our data retention policy. I'll discuss with the team
here - I certainly don't want to muck with RRDtool's innards.
On 11/13/07, Jason Fesler <[EMAIL PROTECTED]> wrote:
> > No way around this limitation?
>
> How good is
> No way around this limitation?
How good is your C programming? :-)
Consider only replaying the last *hour* of data?
(I'm very aware the value of time based data.. in my case only the last
hour is critical, the rest of it, I can afford gaps, as it is trend only).
_
Jason, thanks. We considered that but it's not ideal:
1) Metric data is less valuable over time. If it takes us 2 days to
recover, starting with the oldest data, our customers will be
screaming to know how their systems are performing NOW.
2) The amount of metric data is potentially huge so we n
> older data. If we push current data into an RRD file, is it possible
> to push older data into it since the lastupdated timestamp will be
> greater? Is it possible to fill in the gap without having to send all
> the old data at once?
Nope. SEnd the oldest data first. Maybe don't send it all
We have a monitoring service which collects metrics. If that service
is down, we queue up the stats to be pushed into RRD later. Once the
service comes back online, we want to push the latest metric data
along with the older data (not all at once so as to not overwhelm the
service once it is back
