Re: [rrd-users] Using Granular RRA Automatically

[Simon Hobson]

Stephen R Guglielmo  wrote:

> If I were to do something
> like the following:
> 
> rrdtool create filename.rrd \
>  --step '86400' \
>  'DS:Sites:GAUGE:172800:0:U' \
>  'DS:DeletedSites:GAUGE:172800:0:U' \
>  'DS:OutdatedSites:GAUGE:172800:0:U' \
>  'DS:Users:GAUGE:172800:0:U' \
>  'RRA:LAST:0.3:1:3660'
> 
> This will basically keep every single value entered for 3660 days (~10
> years), correct? It won't consolidate any values?

Correct.

> I understand this
> will have increased disk usage in the .rrd file, but are there any
> other downsides to doing this?

For this application, I don't think so.
In general, it's a trade-off between the amount of data retained (ie how far 
back you can go vs how detailed it is), disk usage, and resources needed to 
draw graphs.
So if you were to keep (say) traffic for every 5 seconds for the last 10 years 
with no consolidation, then drawing graphs would be very I/O and processor 
intensive as it would have to do a lot of consolidation to generate the graph 
(as well as taking a lot of disk space). In your case, even keeping 
unconsolidated data for 10 years, it's not a large amount of data, and unless 
youre system is very resource constrained, I doubt it's going to make any 
difference.

___
rrd-users mailing list
rrd-users@lists.oetiker.ch
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Re: [rrd-users] Using Granular RRA Automatically

[Stephen R Guglielmo]

On Mon, Jan 23, 2017 at 6:06 PM, Simon Hobson  wrote:
> Stephen R Guglielmo  wrote:
>
>> With the above database, I update it once a day with four data points.
>> I record MIN, MAX, AVERAGE, and LAST on a weekly basis for 10 years. I
>> also record MIN, MAX, AVERAGE, and LAST on a daily basis for 35 days.
>>
>> My question is: Since the RRAs are not named, if I create 1 month, 1
>> year, and 2 year graphs, will the 1 month graph automatically use the
>> more granular RRAs (the "daily" 35-day ones as opposed to the "weekly"
>> 10-year ones)?
>
> Yes, it'll automatically use the "best fitting" RRA - which in most cases 
> will be the right thing. You can also specify the resolution in your graph 
> command.
>
>
>> Here is an example `graph` command that I received from the RRD Wizard:
>>
>> rrdtool graph 'sites-1month.png' \
>> --title 'Sites, 1 Month' \
>> --vertical-label 'Number of Sites' \
>> --width '800' \
>> --height '300' \
>> --full-size-mode \
>> --start end-1m \
>> ...
>
>
> It is recommended to explicitly specify the end time, and to round it to an 
> integer multiple of the RRA resolution. So, for an RRA with a resolution of 1 
> day (86400 seconds), specify the end time as an integer multiple of 86400.
> And specify the start time as end-(integer multiple of resolution), and 
> ideally the multiple will be the number of pixels in the graph which avoids 
> rescaling the data to fit.
>
> Longer explanation, RRD will use the RRA which best fits - specifically it'll 
> pick the RRA(s) providing the most data within the specified timescale, and 
> if more than one, the one with the highest resolution. So in your case, if 
> you plot a graph over 36 days then the 1 day RRA won't fill it and it'll 
> switch to a coarser RRA. for 35 days and below, it'll pick the 1 day RRA as 
> having the higher resolution.
> But there are cases where a coarser resolution may provide more data - though 
> I don't think they are very common.
>
> Consider if you aggregated at 2 day and 5 day resolution - contrived but 
> illustrates the point. There will be times where the 5 day RRA covers a day 
> later than the 2 day RRA (basically where it's a multiple of 5 days since 
> unix epoch) - and so the "use the RRA with the most data in the desired 
> period" method will switch to the 5 day RRA.
> For a case like this, you'd need to explicitly specify the RRA to be used, 
> and your data would stop a day earlier while showing more detail.


Simon,

Thank you for the information. It's helpful. If I were to do something
like the following:

rrdtool create filename.rrd \
  --step '86400' \
  'DS:Sites:GAUGE:172800:0:U' \
  'DS:DeletedSites:GAUGE:172800:0:U' \
  'DS:OutdatedSites:GAUGE:172800:0:U' \
  'DS:Users:GAUGE:172800:0:U' \
  'RRA:LAST:0.3:1:3660'

This will basically keep every single value entered for 3660 days (~10
years), correct? It won't consolidate any values? I understand this
will have increased disk usage in the .rrd file, but are there any
other downsides to doing this?

Thank you!
Steve

___
rrd-users mailing list
rrd-users@lists.oetiker.ch
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Re: [rrd-users] Using Granular RRA Automatically

[Simon Hobson]

Stephen R Guglielmo  wrote:

> With the above database, I update it once a day with four data points.
> I record MIN, MAX, AVERAGE, and LAST on a weekly basis for 10 years. I
> also record MIN, MAX, AVERAGE, and LAST on a daily basis for 35 days.
> 
> My question is: Since the RRAs are not named, if I create 1 month, 1
> year, and 2 year graphs, will the 1 month graph automatically use the
> more granular RRAs (the "daily" 35-day ones as opposed to the "weekly"
> 10-year ones)?

Yes, it'll automatically use the "best fitting" RRA - which in most cases will 
be the right thing. You can also specify the resolution in your graph command.


> Here is an example `graph` command that I received from the RRD Wizard:
> 
> rrdtool graph 'sites-1month.png' \
> --title 'Sites, 1 Month' \
> --vertical-label 'Number of Sites' \
> --width '800' \
> --height '300' \
> --full-size-mode \
> --start end-1m \
> ...


It is recommended to explicitly specify the end time, and to round it to an 
integer multiple of the RRA resolution. So, for an RRA with a resolution of 1 
day (86400 seconds), specify the end time as an integer multiple of 86400.
And specify the start time as end-(integer multiple of resolution), and ideally 
the multiple will be the number of pixels in the graph which avoids rescaling 
the data to fit.

Longer explanation, RRD will use the RRA which best fits - specifically it'll 
pick the RRA(s) providing the most data within the specified timescale, and if 
more than one, the one with the highest resolution. So in your case, if you 
plot a graph over 36 days then the 1 day RRA won't fill it and it'll switch to 
a coarser RRA. for 35 days and below, it'll pick the 1 day RRA as having the 
higher resolution.
But there are cases where a coarser resolution may provide more data - though I 
don't think they are very common.

Consider if you aggregated at 2 day and 5 day resolution - contrived but 
illustrates the point. There will be times where the 5 day RRA covers a day 
later than the 2 day RRA (basically where it's a multiple of 5 days since unix 
epoch) - and so the "use the RRA with the most data in the desired period" 
method will switch to the 5 day RRA.
For a case like this, you'd need to explicitly specify the RRA to be used, and 
your data would stop a day earlier while showing more detail.
___
rrd-users mailing list
rrd-users@lists.oetiker.ch
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

[rrd-users] Using Granular RRA Automatically

[Stephen R Guglielmo]

Hello,

I'm new to rrdtool. I'm trying to create a database that will record
and graph data from an application. I have been using the rrdtool
documentation and the RRD Wizard website to assist me. I'd like to
keep track of "total sites," "deleted sites," "outdated sites, and
"total users" (users will be a separate graph).

rrdtool create filename.rrd \
  --step '86400' \
  'DS:TotalSites:GAUGE:172800:0:U' \
  'DS:DeletedSites:GAUGE:172800:0:U' \
  'DS:OutdatedSites:GAUGE:172800:0:U' \
  'DS:TotalUsers:GAUGE:172800:0:U' \
  'RRA:MIN:0.3:7:520' \
  'RRA:MAX:0.3:7:520' \
  'RRA:AVERAGE:0.3:7:520' \
  'RRA:LAST:0.3:7:520' \
  'RRA:MIN:0.3:1:35' \
  'RRA:MAX:0.3:1:35' \
  'RRA:AVERAGE:0.3:1:35' \
  'RRA:LAST:0.3:1:35'

With the above database, I update it once a day with four data points.
I record MIN, MAX, AVERAGE, and LAST on a weekly basis for 10 years. I
also record MIN, MAX, AVERAGE, and LAST on a daily basis for 35 days.

My question is: Since the RRAs are not named, if I create 1 month, 1
year, and 2 year graphs, will the 1 month graph automatically use the
more granular RRAs (the "daily" 35-day ones as opposed to the "weekly"
10-year ones)?

Here is an example `graph` command that I received from the RRD Wizard:

rrdtool graph 'sites-1month.png' \
--title 'Sites, 1 Month' \
--vertical-label 'Number of Sites' \
--width '800' \
--height '300' \
--full-size-mode \
--start end-1m \
'DEF:SitesMin=sites-new.rrd:TotalSites:MIN' \
'DEF:SitesMax=sites-new.rrd:TotalSites:MAX' \
'DEF:SitesAvg=sites-new.rrd:TotalSites:AVERAGE' \
'DEF:SitesLast=sites-new.rrd:TotalSites:LAST' \
'DEF:DeletedMin=sites-new.rrd:DeletedSites:MIN' \
'DEF:DeletedMax=sites-new.rrd:DeletedSites:MAX' \
'DEF:DeletedAvg=sites-new.rrd:DeletedSites:AVERAGE' \
'DEF:DeletedLast=sites-new.rrd:DeletedSites:LAST' \
'DEF:OutdatedMin=sites-new.rrd:OutdatedSites:MIN' \
'DEF:OutdatedMax=sites-new.rrd:OutdatedSites:MAX' \
'DEF:OutdatedAvg=sites-new.rrd:OutdatedSites:AVERAGE' \
'DEF:OutdatedLast=sites-new.rrd:OutdatedSites:LAST' \
'VDEF:SitesMinMin=SitesMin,MINIMUM' \
'VDEF:SitesMaxMax=SitesMax,MAXIMUM' \
'VDEF:SitesAvgAvg=SitesAvg,AVERAGE' \
'VDEF:DeletedMinMin=DeletedMin,MINIMUM' \
'VDEF:DeletedMaxMax=DeletedMax,MAXIMUM' \
'VDEF:OudatedMinMin=OutdatedMin,MINIMUM' \
'VDEF:OutdatedMaxMax=OutdatedMax,MAXIMUM' \
'VDEF:OutdatedAvgAvg=OutdatedAvg,AVERAGE' \
'VDEF:DeletedAvgAvg=DeletedAvg,AVERAGE' \
'LINE1:SitesLast#00FF00:Total Sites' \
'LINE1:DeletedLast#FF:Deleted Sites' \
'LINE1:OutdatedLast#00:Outdated Sites (>=2 Years)' \
'PRINT:SitesMinMin:%.1lf Sites Min' \
'PRINT:SitesMaxMax:%.1lf Sites Max' \
'PRINT:SitesAvgAvg:%.1lf Sites Avg' \
'PRINT:DeletedMinMin:%.1lf Deleted Min' \
'PRINT:DeletedMaxMax:%.1lf Deleted Max' \
'PRINT:DeletedAvgAvg:%.1lf Deleted Avg' \
'PRINT:OudatedMinMin:%.1lf Outdated Min' \
'PRINT:OutdatedMaxMax:%.1lf Outdated Max' \
'PRINT:OutdatedAvgAvg:%.1lf Outdated Avg'


Thank you!
Steve

___
rrd-users mailing list
rrd-users@lists.oetiker.ch
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users