Re: [rrd-users] doubt behavior database RRD

2014-06-26 Thread Steve Shipway 
The problem you have is that by default, when you create the RRD, the last
update time is set to 'now'.   It sets up an initial set of 'unkown' RRA
stretching back intime from the last update time.

 

Then you subsequently use 'rrdtool first' which returns the time of the
START of the RRA, and attempt to update using this.  

 

Of course, it does not work, because this time is in the past, and you can
only update by adding new records with a time after the LAST update.

 

This is probably counterintuitive to you, as you might think you need to
allocate the records that you are intending to update in advance.  In fact,
with RRDtool, you do not do this and records are automatically created and
expired by the RRD mechanism.

 

So, rather than using 'rrdtool first', try using 'rrdtool lastupdate' and
incrementing from that.

 

Steve

 

Steve Shipway

s.ship...@auckland.ac.nz

 

From: rrd-users-bounces+s.shipway=auckland.ac...@lists.oetiker.ch
[mailto:rrd-users-bounces+s.shipway=auckland.ac...@lists.oetiker.ch] On
Behalf Of spiderslack
Sent: Friday, 27 June 2014 8:23 a.m.
To: rrd-users@lists.oetiker.ch
Subject: [rrd-users] doubt behavior database RRD

 

Hi all.

i testing the database rrd. I create the shellscript follow bellow:


--- test.sh --
set -x
ARQUIVO_RRD="kakaroto.rrd"
rm $ARQUIVO_RRD
rrdtool create $ARQUIVO_RRD \
--start N --step 15 \
DS:dados:COUNTER:20:U:U   \
RRA:AVERAGE:0.5:1:60   \


date +%s
INICIO=$( rrdtool first $ARQUIVO_RRD )
ATUAL=$INICIO
sleep 15
INICIO=$( expr $INICIO + 15)
rrdtool update $ARQUIVO_RRD $INICIO:10
---

when execute the shell script


[ricardobarbosa@shark transito]$ bash .sh
+ ARQUIVO_RRD=kakaroto.rrd
+ rm kakaroto.rrd
+ rrdtool create kakaroto.rrd --start N --step 15 DS:dados:COUNTER:20:U:U
RRA:AVERAGE:0.5:1:60
+ date +%s
1403813403
++ rrdtool first kakaroto.rrd
+ INICIO=1403812515
+ ATUAL=1403812515
+ sleep 15
++ expr 1403812515 + 15
+ INICIO=1403812530
+ rrdtool update kakaroto.rrd 1403812530:10
ERROR: kakaroto.rrd: illegal attempt to update using time 1403812530 when
last update time is 1403813403 (minimum one second step)
[ricardobarbosa@shark transito]$
---
[ricardobarbosa@shark transito]$ bash .sh
+ ARQUIVO_RRD=kakaroto.rrd
+ rm kakaroto.rrd
+ rrdtool create kakaroto.rrd --start N --step 15 DS:dados:COUNTER:20:U:U
RRA:AVERAGE:0.5:1:60
+ date +%s
1403813422
++ rrdtool first kakaroto.rrd
+ INICIO=1403812530
+ ATUAL=1403812530
+ sleep 15
++ expr 1403812530 + 15
+ INICIO=1403812545
+ rrdtool update kakaroto.rrd 1403812545:10
ERROR: kakaroto.rrd: illegal attempt to update using time 1403812545 when
last update time is 1403813422 (minimum one second step)
[ricardobarbosa@shark transito]$




The value th output command (rrdtool first) is correct? should not be the
current epoch, since it was not updated anything in the file. rrd?

in the last example the timestamp used when creating the rrd file is
1403813422, but the output of command (rrdtool first) he shows how the
timestamp 1403812530 is smaller. The value of the output of the command
(rrdtool first) should not be at least 1403813XXX.



each execution of the script I delete the rrd file. he should not reset? it
seems that he is guarding the first of the previous file. If someone can
help me understand I thank.


Regards.





smime.p7s
Description: S/MIME cryptographic signature
___
rrd-users mailing list
rrd-users@lists.oetiker.ch
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

[rrd-users] doubt behavior database RRD

2014-06-26 Thread spiderslack 

Hi all.

i testing the database rrd. I create the shellscript follow bellow:


--- test.sh --
set -x
ARQUIVO_RRD="kakaroto.rrd"
rm $ARQUIVO_RRD
rrdtool create $ARQUIVO_RRD \
--start N --step 15 \
DS:dados:COUNTER:20:U:U   \
RRA:AVERAGE:0.5:1:60   \


date +%s
INICIO=$( rrdtool first $ARQUIVO_RRD )
ATUAL=$INICIO
sleep 15
INICIO=$( expr $INICIO + 15)
rrdtool update $ARQUIVO_RRD $INICIO:10
---

when execute the shell script


[ricardobarbosa@shark transito]$ bash .sh
+ ARQUIVO_RRD=kakaroto.rrd
+ rm kakaroto.rrd
+ rrdtool create kakaroto.rrd --start N --step 15 
DS:dados:COUNTER:20:U:U RRA:AVERAGE:0.5:1:60

+ date +%s
1403813403
++ rrdtool first kakaroto.rrd
+ INICIO=1403812515
+ ATUAL=1403812515
+ sleep 15
++ expr 1403812515 + 15
+ INICIO=1403812530
+ rrdtool update kakaroto.rrd 1403812530:10
ERROR: kakaroto.rrd: illegal attempt to update using time 1403812530 
when last update time is 1403813403 (minimum one second step)

[ricardobarbosa@shark transito]$
---
[ricardobarbosa@shark transito]$ bash .sh
+ ARQUIVO_RRD=kakaroto.rrd
+ rm kakaroto.rrd
+ rrdtool create kakaroto.rrd --start N --step 15 
DS:dados:COUNTER:20:U:U RRA:AVERAGE:0.5:1:60

+ date +%s
1403813422
++ rrdtool first kakaroto.rrd
+ INICIO=1403812530
+ ATUAL=1403812530
+ sleep 15
++ expr 1403812530 + 15
+ INICIO=1403812545
+ rrdtool update kakaroto.rrd 1403812545:10
ERROR: kakaroto.rrd: illegal attempt to update using time 1403812545 
when last update time is 1403813422 (minimum one second step)

[ricardobarbosa@shark transito]$




The value th output command (rrdtool first) is correct?should not be the 
current epoch, since it was not updated anything in the file. rrd?


in the last example the timestamp used when creating the rrd file is 
1403813422, but the output of command (rrdtool first) he shows how the 
timestamp 1403812530 is smaller. The value of the output of the command 
(rrdtool first) should not be at least 1403813XXX.


each execution of the script I delete the rrd file. he should not reset? 
it seems that he is guarding the first of the previous file. If someone 
can help me understand I thank.


Regards.
___
rrd-users mailing list
rrd-users@lists.oetiker.ch
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Re: [rrd-users] wrong rra used

2014-06-26 Thread Simon Hobson 
spock  wrote:

> Do you see any trick to create a single data point per day with the average
> value (temperature)?

You could create two RRDs - one with and one without a high resolution 
consolidation. Then you can use the two RRDs in the one graph.
Either modify your scripts to update both RRDs, or have another script that 
updates the second with data from the first. I'd recommend the former, the 
latter can be messy.



___
rrd-users mailing list
rrd-users@lists.oetiker.ch
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Re: [rrd-users] wrong rra used

2014-06-26 Thread spock 
Hello Simon,

thanks for your comments...

> It does the best it can to use the most detailed data. Looking at the
> docs, I don't see any
> option to override that. 

I saw a similar setup on  Link  
and there are also a 2 RRA; the example graph showed the correct average
line: average of all values of a single day.
Maybe I try to contact him if there is any "trick" involved.

Do you see any trick to create a single data point per day with the average
value (temperature)?


> Looks more like an hour to me, are you on +1 timezone by any chance ? 
Yes, I am in UTC+1 timezone...
So data consolidation is always done based on UTC+0 timezone.

I am using the following unix statements to get the timestamp:

set $(date "+%Y %m %d %H %M")
m=$((100+15*(${5#0}/15)))
timestamp_rounded="$1$2$3 $4:${m#1}"
timestamp_rounded_unix=$(date --date "$1$2$3 $4:${m#1}" +%s)
timestamp_real=$(date "+%Y%m%d %H:%M")
timestamp_real_unix=$(date --date "$timestamp_real" +%s)

If I check my own logfiles, i can see:
I106 rrdtool update /testtemp/temp_pool.rrd
1403702100:23.5:22.6:29.8:23.5:23.9:1:1:50.8
I103 20140625 15:15 --> this is my time, UTC+1 CEST

1403702100 translates into 25.06.2014 - 15:15:00

So the algorithm from a rrdtool perspective uses my time as UTC+0 time.

I still cannot explain this 1h offset in the graph...



> Are you certain you are updating frequently enough ? 

The reason here was a downtime due to maintenance work.
Well, as I understand it, the RRA function tolerates with SSF 0.5, that 50%
of data points may be missing. As you can see in the graph, there are a few
values missing, but not 50% (48 data points out of 96).  Or does the max
function return no values, if the data vector contains unknowns?




--
View this message in context: 
http://rrd-mailinglists.937164.n2.nabble.com/wrong-rra-used-tp7582214p7582216.html
Sent from the RRDtool Users Mailinglist mailing list archive at Nabble.com.

___
rrd-users mailing list
rrd-users@lists.oetiker.ch
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Re: [rrd-users] wrong rra used

2014-06-26 Thread Simon Hobson 
spock  wrote:

> I am using this database to collect temperature data for a swimming pool.
> I want to keep the detailed data for 36 months ("103680 * 900sec") but also
> want to have a daily average temp (96 values = 24h).
> 
> # rrdtool create temp_pool.rrd --step 900 --start "20140101 00:00" \
> # DS:air:GAUGE:1200:-40:50 \
> # DS:flow:GAUGE:1200:-40:50 \
> # DS:return:GAUGE:1200:-40:50 \
> # DS:delta:COMPUTE:return,flow,- \
> # DS:pump_flag:GAUGE:1200:0:1 \
> # DS:heat_flag:GAUGE:1200:0:1 \
> # DS:device:GAUGE:1200:0:90
> # RRA:AVERAGE:0.5:1:103680 \
> # RRA:MIN:0.5:96:3650 \
> # RRA:MAX:0.5:96:3650 \
> # RRA:AVERAGE:0.5:96:3650
> 
> When I create a graph with 
>   rrdtool graph temp7d.png \
>-t "Pooltemp (Status $(date "+%d.%m.%Y %H:%M"))"  \
>-s $graph_start_7d -e $timestamp_rounded_unix \
>-w 640 -h 400 -D -a PNG -T 15\
>DEF:airmax2=temp_pool.rrd:air:MAX \
>DEF:airmin2=temp_pool.rrd:air:MIN \
>DEF:airavg2=temp_pool.rrd:air:AVERAGE \
>CDEF:airrange=airmax2,airmin2,- \
>LINE2:airmin2#00FF00:"Luft Min" \
>AREA:airrange#8dadf588::STACK \
>LINE2:airmax2#00FF00:"Luft Min" \
>LINE2:airavg2#00FF00:Aussen
> 
> I get for airavg2 the datailed values - but not the daily average.
> Airmin2 and airmax2 work as  expected - there is no other RRA Archive to
> choose from.
> 
> Question (1)
> How do I convince the tool to use the consolidated data instead of the
> detailed?

It does the best it can to use the most detailed data. Looking at the docs, I 
don't see any option to override that.


> Question (2)
> When drawing the graph, the line for max/min values are not precisely
> aligned above a day. It looks like it starts about 900 secs after 0:00 each
> day.

Looks more like an hour to me, are you on +1 timezone by any chance ?

> How does rrdtool determine, which 96 values to consolidate? My data point
> are stored with rounded timestamps (e.g. 13.06.2014 0:00, 0:15, 0:30, etc.)

*ALL* periods are aligned to an integer multiple of step (or step * steps/CDP) 
since unix epoch (midnight 1-1-1970 UTC). So if you consolidate to a day then a 
"day" will always start at 00:00:00 UTC. If your timezone isn't UTC, then the 
data will appear to be off by your timezone offset.


> Question (3)
> The ssf factor is 0.5 for the RRAs. However, there is no min/max value for
> certain days with missing data, although there are only a few values
> missing. I am sure, is is not more than 50%. How can I check this.
> 
>  

Are you certain you are updating frequently enough ? I can see some gaps in the 
data, if you have more than 1200s between updates then you'll have gaps.


For more on how consolidation is done, see this tutorial
http://www.vandenbogaerdt.nl/rrdtool/process.php

___
rrd-users mailing list
rrd-users@lists.oetiker.ch
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

[rrd-users] wrong rra used

2014-06-26 Thread spock 
Hi, 

I am using this database to collect temperature data for a swimming pool.
I want to keep the detailed data for 36 months ("103680 * 900sec") but also
want to have a daily average temp (96 values = 24h).

# rrdtool create temp_pool.rrd --step 900 --start "20140101 00:00" \
# DS:air:GAUGE:1200:-40:50 \
# DS:flow:GAUGE:1200:-40:50 \
# DS:return:GAUGE:1200:-40:50 \
# DS:delta:COMPUTE:return,flow,- \
# DS:pump_flag:GAUGE:1200:0:1 \
# DS:heat_flag:GAUGE:1200:0:1 \
# DS:device:GAUGE:1200:0:90
# RRA:AVERAGE:0.5:1:103680 \
# RRA:MIN:0.5:96:3650 \
# RRA:MAX:0.5:96:3650 \
# RRA:AVERAGE:0.5:96:3650

When I create a graph with 
   rrdtool graph temp7d.png \
-t "Pooltemp (Status $(date "+%d.%m.%Y %H:%M"))"  \
-s $graph_start_7d -e $timestamp_rounded_unix \
-w 640 -h 400 -D -a PNG -T 15\
DEF:airmax2=temp_pool.rrd:air:MAX \
DEF:airmin2=temp_pool.rrd:air:MIN \
DEF:airavg2=temp_pool.rrd:air:AVERAGE \
CDEF:airrange=airmax2,airmin2,- \
LINE2:airmin2#00FF00:"Luft Min" \
AREA:airrange#8dadf588::STACK \
LINE2:airmax2#00FF00:"Luft Min" \
LINE2:airavg2#00FF00:Aussen

I get for airavg2 the datailed values - but not the daily average.
Airmin2 and airmax2 work as  expected - there is no other RRA Archive to
choose from.

Question (1)
How do I convince the tool to use the consolidated data instead of the
detailed?

Question (2)
When drawing the graph, the line for max/min values are not precisely
aligned above a day. It looks like it starts about 900 secs after 0:00 each
day.
How does rrdtool determine, which 96 values to consolidate? My data point
are stored with rounded timestamps (e.g. 13.06.2014 0:00, 0:15, 0:30, etc.)

Question (3)
The ssf factor is 0.5 for the RRAs. However, there is no min/max value for
certain days with missing data, although there are only a few values
missing. I am sure, is is not more than 50%. How can I check this.

 



--
View this message in context: 
http://rrd-mailinglists.937164.n2.nabble.com/wrong-rra-used-tp7582214.html
Sent from the RRDtool Users Mailinglist mailing list archive at Nabble.com.

___
rrd-users mailing list
rrd-users@lists.oetiker.ch
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users