Re: [rrd-users] trying to understand the relationship between source data, what's in rrd and what gets plotted

2007-07-25 Thread Mark Seger
Simon Hobson wrote: > Mark Seger wrote: > > >> > That is because hh:mm:06, hh:mm:16, hh:mm:26 and so on are not a whole >> >>> multiple of 10 seconds. >>> >>> You have "n*step+offset", not "n*step". This is why normalization is >>> needed. >>> >>> >>> >>> As I said abo

Re: [rrd-users] trying to understand the relationship between source data, what's in rrd and what gets plotted

2007-07-25 Thread Alex van den Bogaerdt
On Wed, Jul 25, 2007 at 08:57:32AM -0400, Mark Seger wrote: > so if I understand what you're suggesting I should pick a start time and > step size such that my data will align accordingly, right? Since I have > samples at 00:01:06, 01:12, etc that would mean I should pick a time > that lands o

Re: [rrd-users] trying to understand the relationship between source data, what's in rrd and what gets plotted

2007-07-25 Thread Simon Hobson
Mark Seger wrote: > > That is because hh:mm:06, hh:mm:16, hh:mm:26 and so on are not a whole >> multiple of 10 seconds. >> >> You have "n*step+offset", not "n*step". This is why normalization is >> needed. >> >> >> >>> As I said above it sounds like if I conform my data to align to the tim

Re: [rrd-users] trying to understand the relationship between source data, what's in rrd and what gets plotted

2007-07-25 Thread Mark Seger
Alex van den Bogaerdt wrote: > On Wed, Jul 25, 2007 at 07:43:50AM -0400, Mark Seger wrote: > > >>> "Nearest minute boundary": No, it does not. You tell it where to >>> do so, using "--step". >>> >>> >> Maybe I misled you by the term 'minute boundary', but I meant that it >> appear

Re: [rrd-users] trying to understand the relationship between source data, what's in rrd and what gets plotted

2007-07-25 Thread Alex van den Bogaerdt
On Wed, Jul 25, 2007 at 07:43:50AM -0400, Mark Seger wrote: > > "Nearest minute boundary": No, it does not. You tell it where to > > do so, using "--step". > > > Maybe I misled you by the term 'minute boundary', but I meant that it > appeared to align the data such that at least one sample f

Re: [rrd-users] trying to understand the relationship between source data, what's in rrd and what gets plotted

2007-07-25 Thread Mark Seger
Alex van den Bogaerdt wrote: >> In any event, when I look at the contents of the rrd database that >> contains a day's worth of 10 second samples using fetch. starting at >> 1123992076 I see the first interval at 1123992080, which leads me to a >> couple of questions: >> - does rrd choose to

Re: [rrd-users] trying to understand the relationship between source data, what's in rrd and what gets plotted

2007-07-25 Thread Alex van den Bogaerdt
> In any event, when I look at the contents of the rrd database that > contains a day's worth of 10 second samples using fetch. starting at > 1123992076 I see the first interval at 1123992080, which leads me to a > couple of questions: > - does rrd choose to normalize data to the nearest minute

Re: [rrd-users] trying to understand the relationship between source data, what's in rrd and what gets plotted

2007-07-24 Thread Simon Hobson
Mark Seger wrote: >I tried what you suggested by using a min/max RRA in addition to storing >all the data in a 3rd RRA and it appears to work as you suggested it >would, but some of the peaks aren't as high as I expected and so I did >some further digging to try and understand what I'm seeing. I h

Re: [rrd-users] trying to understand the relationship between source data, what's in rrd and what gets plotted

2007-07-24 Thread Mark Seger
I tried what you suggested by using a min/max RRA in addition to storing all the data in a 3rd RRA and it appears to work as you suggested it would, but some of the peaks aren't as high as I expected and so I did some further digging to try and understand what I'm seeing. I have samples that w

Re: [rrd-users] trying to understand the relationship between source data, what's in rrd and what gets plotted

2007-07-22 Thread Alex van den Bogaerdt
On Sun, Jul 22, 2007 at 07:23:44AM -0400, Mark Seger wrote: > I think this makes a lot of sense and I'm certainly willing to give it a > shot and see what the results look like as you can never be completely > sure until you try it. So the next obvious question becomes how much > work is it to

Re: [rrd-users] trying to understand the relationship between source data, what's in rrd and what gets plotted

2007-07-22 Thread Mark Seger
I think this makes a lot of sense and I'm certainly willing to give it a shot and see what the results look like as you can never be completely sure until you try it. So the next obvious question becomes how much work is it to do something like this in and more important how much overhead will

Re: [rrd-users] trying to understand the relationship between source data, what's in rrd and what gets plotted

2007-07-22 Thread Alex van den Bogaerdt
On Sat, Jul 21, 2007 at 02:52:55PM -0400, Mark Seger wrote: > >> Just put the dots (or lines) where they belong and if some fall on top > >> of each other and you only see 6, that's just fine. Where I'm coming > >> from, and what I always do with performance related data, is look at a > >> bro

Re: [rrd-users] trying to understand the relationship between source data, what's in rrd and what gets plotted

2007-07-21 Thread Simon Hobson
Mark Seger wrote: >I think what rrd shines at is being able to archive data at very coarse >time intervals, ultimately being able to show you things like average >loads, etc where you're really only interested in trends and I want to >look at diagnostic data and never lose a data point. Well you

Re: [rrd-users] trying to understand the relationship between source data, what's in rrd and what gets plotted

2007-07-21 Thread Mark Seger
Alex van den Bogaerdt wrote: > On Sat, Jul 21, 2007 at 09:42:47AM -0400, Mark Seger wrote: > > >> Just put the dots (or lines) where they belong and if some fall on top >> of each other and you only see 6, that's just fine. Where I'm coming >> from, and what I always do with performance rel

Re: [rrd-users] trying to understand the relationship between source data, what's in rrd and what gets plotted

2007-07-21 Thread Alex van den Bogaerdt
On Sat, Jul 21, 2007 at 09:42:47AM -0400, Mark Seger wrote: > Just put the dots (or lines) where they belong and if some fall on top > of each other and you only see 6, that's just fine. Where I'm coming > from, and what I always do with performance related data, is look at a > broad range of

Re: [rrd-users] trying to understand the relationship between source data, what's in rrd and what gets plotted

2007-07-21 Thread Mark Seger
> Please let me summarize in my own words: > > If you display 4000 rates on a 400-pixel wide graph, you want to > see 10 dots per column, not a line or an area connecting an > minimum, average, maximum or last. > > Correct? > correct, but a line can be useful too! > If so: > > This would requir

Re: [rrd-users] trying to understand the relationship between source data, what's in rrd and what gets plotted

2007-07-21 Thread Alex van den Bogaerdt
On Sat, Jul 21, 2007 at 08:03:34AM -0400, Mark Seger wrote: > yes, I agree completely about the ability about rrd's strength being to > be able to store data long term. but if take a sample every 5 seconds > but then can't look at it at that granularity what's the point? I did > see the sugge

Re: [rrd-users] trying to understand the relationship between source data, what's in rrd and what gets plotted

2007-07-21 Thread Mark Seger
Simon Hobson wrote: > Mark Seger wrote: > >> The thing that's interesting about this whole situation is that on one >> level rrd appears to draw a cleaner graph and the gnuplot one looks a >> little fuzzier, but I also think the gnuplot provides valuable >> information that gets lost, and prob

Re: [rrd-users] trying to understand the relationship between source data, what's in rrd and what gets plotted

2007-07-21 Thread Simon Hobson
Mark Seger wrote: >The thing that's interesting about this whole situation is that on one >level rrd appears to draw a cleaner graph and the gnuplot one looks a >little fuzzier, but I also think the gnuplot provides valuable >information that gets lost, and probably missed with rrd. If my >example

Re: [rrd-users] trying to understand the relationship between source data, what's in rrd and what gets plotted

2007-07-21 Thread Alex van den Bogaerdt
On Sat, Jul 21, 2007 at 08:41:47AM +0200, Tobias Oetiker wrote: > yes, as I pointed out in my last reply, you can use 3 RRAs with > MIN, AVERAGE and MAX consolidation and then plot the data such dat > you draw an area between MIN and MAX and overlay the AVERAGE line. > > The approach is to use CD

Re: [rrd-users] trying to understand the relationship between source data, what's in rrd and what gets plotted

2007-07-20 Thread Tobias Oetiker
Mark, yes, as I pointed out in my last reply, you can use 3 RRAs with MIN, AVERAGE and MAX consolidation and then plot the data such dat you draw an area between MIN and MAX and overlay the AVERAGE line. The approach is to use CDEF to build the difference between MIN and MAX, then draw MIN withou

Re: [rrd-users] trying to understand the relationship between source data, what's in rrd and what gets plotted

2007-07-20 Thread Mark Seger
The thing that's interesting about this whole situation is that on one level rrd appears to draw a cleaner graph and the gnuplot one looks a little fuzzier, but I also think the gnuplot provides valuable information that gets lost, and probably missed with rrd. If my examples were disk perform

Re: [rrd-users] trying to understand the relationship between source data, what's in rrd and what gets plotted

2007-07-20 Thread Tobias Oetiker
Hi Mark, I will add the following to the rrdcreate manpage =over =item AVERAGE the average of the data points is stored. =item MIN the smallest of the data points is stored. =item MAX the largest of the data points is stored. =item LAST the last data points is used. =back Note that

Re: [rrd-users] trying to understand the relationship between source data, what's in rrd and what gets plotted

2007-07-20 Thread Tobias Oetiker
Mark, think about this: you give rrdtool 3 samples with the following values: 10, 100, 990 now you ask rrdtool to plot the data. And ithappens all three values get mapped to 1 pixel. If you pick AVERAGE the result will be 100 If you pick MAX it will be 990 If you pick MIN it will be 10 What

Re: [rrd-users] trying to understand the relationship between source data, what's in rrd and what gets plotted

2007-07-20 Thread Mark Seger
ok, I really hate to be a pain but I really want to get this right too and have spent almost 2 solid days trying to understand this and am still puzzled! I think rrd is pretty cool but if it's not the right tool to be using, just tell me. I believe when doing system monitoring it's absolutely

Re: [rrd-users] trying to understand the relationship between source data, what's in rrd and what gets plotted

2007-07-20 Thread Mark Seger
I'd say my problems arose from the fact that I could find any description on what average, min, max and last do! If they're already somewhere may all that's needed is a link, but I didn't see anything. That said, under create manpage under RRA I'd expand the description of the consolidate fun

Re: [rrd-users] trying to understand the relationship between source data, what's in rrd and what gets plotted

2007-07-20 Thread Tobias Oetiker
Hi Mark, yes the 'lost' spike confuses people ... most, when they start thinking about it, see that rrdtool does exactly the right thing, it uses to consolidation method of the data being graphed to further consolidate for the graph ... so ifyou are using MAX as consolidation function for the RRA

Re: [rrd-users] trying to understand the relationship between source data, what's in rrd and what gets plotted

2007-07-20 Thread Mark Seger
Alex van den Bogaerdt wrote: > On Fri, Jul 20, 2007 at 12:31:25PM -0400, Mark Seger wrote: > >> more experiments and I'm getting closer... I think the problem is the >> AVERAGE in my DEF statements of the graphing command. The only problem >> is I couldn't find any clear description or exa

Re: [rrd-users] trying to understand the relationship between source data, what's in rrd and what gets plotted

2007-07-20 Thread Alex van den Bogaerdt
On Fri, Jul 20, 2007 at 12:31:25PM -0400, Mark Seger wrote: > more experiments and I'm getting closer... I think the problem is the > AVERAGE in my DEF statements of the graphing command. The only problem > is I couldn't find any clear description or examples of how this works. > I did try us

Re: [rrd-users] trying to understand the relationship between source data, what's in rrd and what gets plotted

2007-07-20 Thread Mark Seger
more experiments and I'm getting closer... I think the problem is the AVERAGE in my DEF statements of the graphing command. The only problem is I couldn't find any clear description or examples of how this works. I did try using LAST (even though I have no idea what it does) and my plots got

Re: [rrd-users] trying to understand the relationship between source data, what's in rrd and what gets plotted

2007-07-20 Thread Mark Seger
thanks for the detailed reply and the good news is it more or less agrees with how I thought this works. in fact, I did find a bug in my table loading script which caused the table to get created with an interval of 3 seconds vs 1 and so yes, things got normalized in unexpected way. once I go

Re: [rrd-users] trying to understand the relationship between source data, what's in rrd and what gets plotted

2007-07-20 Thread Simon Hobson
Mark Seger wrote: >I'm sure it's me and not rrdtool, but I can't figure out what's >happening! Here's what did: >- I have a data that has been generated at 1 second intervals >- generated a plot with gnuplot as a known quantity >- loaded the same data into rrdtool as absolute values and also with

[rrd-users] trying to understand the relationship between source data, what's in rrd and what gets plotted

2007-07-20 Thread Mark Seger
I'm sure it's me and not rrdtool, but I can't figure out what's happening! Here's what did: - I have a data that has been generated at 1 second intervals - generated a plot with gnuplot as a known quantity - loaded the same data into rrdtool as absolute values and also with an interval of 1 sec