[rrd-users] Re: rrdtool without accurate time
On Thu, Nov 30, 2006 at 02:30:40PM -0800, Joel Lindsay wrote: Hi Joel. Hi, I have no clock on my embedded system so am having problem using rrdtool correctly. I can add a fake date and therefore enter info into the RRD but Not having a time source is usually a bad thing (tm). Can you use NTP on this embedded system? Than you can do a first ntpdate to set the real date and time and run ntpd after this to eliminate clock drifts, etc. So long, Andreas. -- Dipl.-Ing. Andreas Maus science+computing ag System Administration Hagellocher Weg 73 mail: [EMAIL PROTECTED] 72070 Tuebingen, Germany tel.: +49 7071 9457 456 www.science-computing.de -- Binary/unsupported file stripped by Ecartis -- -- Err : No filename to use for decode, file stripped. -- Type: application/pgp-signature -- Unsubscribe mailto:[EMAIL PROTECTED] Helpmailto:[EMAIL PROTECTED] Archive http://lists.ee.ethz.ch/rrd-users WebAdminhttp://lists.ee.ethz.ch/lsg2.cgi
[rrd-users] Re: Announce RRDtool Performance Tester
Hi Tobi/Johannes I think a standard would be a good thing so that we can compare. However when posting stats it would be usefull to know the types of disk and raid configuration as I think these are as important as raw CPU power. Tobi I assume the number of RRA's in the RRD is important as well, I'm not sure though what the majority of people are using, I for example are using RRA:AVERAGE:0.5:1:129600, RRA:AVERAGE:0.5:10:38880, RRA:AVERAGE:0.5:30:52560, RRA:MAX:0.5:1:129600, RRA:MAX:0.5:10:38880, RRA:MAX:0.5:30:52560, but I expect this is not the norm. Getting that close to what the majority of people are using would make the test more relevant to the real world but I'm not sure if making such an assumption will be possible. Kind Regards Richard Burton -- Richard Burton at Atomwide Ltd Tel 0870 236 5000 Fax 0870 236 5001 Unit 2, Ravensquay Business Centre, Cray Avenue, Orpington, Kent, BR5 4BQ Mailto:[EMAIL PROTECTED] http://www.atomwide.com/ -Original Message- From: Tobias Oetiker [mailto:[EMAIL PROTECTED] Sent: 30 November 2006 19:31 To: Johannes Huettemeister Cc: Richard Burton; rrd-users@list.ee.ethz.ch; rrd-developers@list.ee.ethz.ch Subject: Re: [rrd-users] Re: Announce RRDtool Performance Tester Hi Johannes, Your system does not seem to experiance that 'performance death' others see ... figures with that much ram ... you may want to try the new version of the script I am attaching ... it tries harder to figure where the limits lie ... as comparisons go I guess we need to define some 'benchmark' what about the following: How many rrdfiles can your maintain when you run at a 1s interval ... how many at a 10s interval and how many at a 60s interval ... in your list below my guess is that your system would be able to keep 2000 rrds up todate ina 1s interval ... cheers tobi Hi Tobi, Sun V440 (4cpus, 16GB RAM, Solaris 10) Create 10 rrds 1 c/s (0.00291 sdv) Update 10 rrds4458 u/s (0.2 sdv) Create 10 rrds 1 c/s (0.00274 sdv) Update 20 rrds4363 u/s (0.3 sdv) Create 20 rrds 1 c/s (0.00538 sdv) Update 40 rrds4218 u/s (0.5 sdv) Create 40 rrds 1 c/s (0.01087 sdv) Update 80 rrds4073 u/s (0.9 sdv) Create 80 rrds 1 c/s (0.02177 sdv) Update160 rrds3967 u/s (0.00013 sdv) Create160 rrds 1 c/s (0.11104 sdv) Update320 rrds3800 u/s (0.00019 sdv) Create320 rrds 5 c/s (0.08539 sdv) Update640 rrds3508 u/s (0.00026 sdv) Create640 rrds 1 c/s (0.24887 sdv) Update 1280 rrds3008 u/s (0.00039 sdv) Create 1280 rrds 2 c/s (0.29471 sdv) Update 2560 rrds2000 u/s (0.00118 sdv) Create 2560 rrds 1 c/s (0.29512 sdv) Update 5120 rrds 816 u/s (0.00225 sdv) Create 1536 rrds 2 c/s (0.30246 sdv) Update 6656 rrds2140 u/s (0.00072 sdv) Can someone give an interpretation? I don't know what to think of my system now:-) cheers Johannes. -- Unsubscribe mailto:[EMAIL PROTECTED] Helpmailto:[EMAIL PROTECTED] Archive http://lists.ee.ethz.ch/rrd-users WebAdminhttp://lists.ee.ethz.ch/lsg2.cgi
[rrd-users] Re: Compromised value after NaN
Thanks for spending time on my problem. According to your answer I tried
to simplify my request.
Alex van den Bogaerdt wrote:
On Wed, Nov 29, 2006 at 12:35:01PM +0100, Markus Wiget wrote:
step 300, heartbeat 600. This means that if the updates would be
exactly five minutes apart, with the exception of that single one,
you should see no gap, at all.
For simplifying the discussion I also changed the relation between step
and heartbeat in the example (below).
Why don't you create a script that shows the problem? I mean a
complete script, creating a (small!) database, doing (a few!) updates
and showing the resulting data with rrdtool fetch (!)
Ok. Here is a Bash script which shows the problem. I tried with RRDtool
1.2.15.
echo '* Create rrdnan.rrd'
rrdtool create rrdnan.rrd --step=5 \
DS:testli:GAUGE:7:0:100 \
RRA:AVERAGE:0.5:1:16
echo '* Feed data'
declare -i i
declare s_date
for i in $(seq 1 10); do
sleep 5
s_date=N
if [ ${i} -ne 6 ]; then
rrdtool update rrdnan.rrd --template=testli ${s_date}:100
echo (${i} / $(date '+%H:%M:%S')) Updated ${s_date}:100
else
echo (${i} / $(date '+%H:%M:%S')) NOP
fi
done
echo '* Fetch data'
rrdtool fetch rrdnan.rrd AVERAGE --start=-60
echo '* Finished'
My output is:
* Create rrdnan.rrd
* Feed data
(1 / 10:55:20) Updated N:100
(2 / 10:55:25) Updated N:100
(3 / 10:55:30) Updated N:100
(4 / 10:55:35) Updated N:100
(5 / 10:55:40) Updated N:100
(6 / 10:55:45) NOP
(7 / 10:55:50) Updated N:100
(8 / 10:55:55) Updated N:100
(9 / 10:56:00) Updated N:100
(10 / 10:56:05) Updated N:100
* Fetch data
testli
1164966910: nan
1164966915: nan
1164966920: nan
1164966925: 9.795288e+01
1164966930: 1.00e+02
1164966935: 1.00e+02
1164966940: 1.00e+02
1164966945: nan
1164966950: nan
1164966955: 9.665760e+01
1164966960: 1.00e+02
1164966965: 1.00e+02
1164966970: nan
* Finished
Use timestamps, not N.
What's wrong when using N? I just read in the man page that it is
legal. Anyway, if I use explicit update time by replacing
s_date=N
with
s_date=$(date '+%s')
then I get this output:
[...]
* Fetch data
testli
1164967090: nan
1164967095: nan
1164967100: nan
1164967105: 1.00e+02
1164967110: 1.00e+02
1164967115: 1.00e+02
1164967120: 1.00e+02
1164967125: nan
1164967130: nan
1164967135: 1.00e+02
1164967140: 1.00e+02
1164967145: 1.00e+02
1164967150: nan
* Finished
So it seems that we have two discussion issues:
#1 Why is the first value after the gap not 100 when updating with
implicit timestamp N (first output)?
But what I do not understand: the first
value after the NaN's is 9.99...+01, but I would expect 1.00...+02.
It looks as if your NaN is somehow being mangled into zero. If this
is indeed the case, I can see why your data would be corrupted.
#2 Why are there two NaN's?
So around 2:30 when the cron job is ommitted, I have two NaN's in the
RRD file; this I understand.
I don't. Please explain why this would be.
I thought it is because RRD always requires two values to build a row:
the current and the previous value. This would mean for the first NaN,
the current is missing and for the second NaN the previous is missing.
But probably I'm totally wrong.
Again, I would be grateful for your support.
Thanks,
Markus Wiget
--
---
addr://Kasinostrasse 30, CH-5001 Aarau fon://++41 62 823 9355
http://www.terreactive.com fax://++41 62 823 9356
---
10 Jahre Kompetenz in IT-Sicherheit.1996 - 2006
Wir sichern Ihren Erfolg.terreActive AG
---
--
Unsubscribe mailto:[EMAIL PROTECTED]
Helpmailto:[EMAIL PROTECTED]
Archive http://lists.ee.ethz.ch/rrd-users
WebAdminhttp://lists.ee.ethz.ch/lsg2.cgi
[rrd-users] Re: Compromised value after NaN
On Fri, Dec 01, 2006 at 01:11:38PM +0100, Markus Wiget wrote:
Use timestamps, not N.
What's wrong when using N? I just read in the man page that it is
legal. Anyway, if I use explicit update time by replacing
So what? Did I say It is illegal to use 'N' ? No, I didn't.
Here's why I asked:
* Create rrdnan.rrd
* Feed data
(1 / 14:56:37) Updated N:100
(2 / 14:56:42) Updated N:100
(3 / 14:56:47) Updated N:100
(4 / 14:56:52) Updated N:100
(5 / 14:56:57) Updated N:100
(6 / 14:57:02) NOP
(7 / 14:57:07) Updated N:100
(8 / 14:57:12) Updated N:100
(9 / 14:57:17) Updated N:100
(10 / 14:57:22) Updated N:100
* Fetch data
testli
1164981385: nan
1164981390: nan
1164981395: nan
1164981400: 8.618757e+01
1164981405: 1.00e+02
1164981410: 1.00e+02
1164981415: 1.00e+02
1164981420: nan
1164981425: nan
1164981430: 1.1806045000e+02
1164981435: 1.00e+02
1164981440: 1.00e+02
1164981445: nan
* Finished
As you can see, I get different values, because we used different
times. That happens because you use 'N' !
(side note: 1.1806045000e+02 seems a weird value. I am going to look into this)
Anyway, if I use explicit update time by replacing
s_date=N
with
s_date=$(date '+%s')
then I get this output:
[...]
1164967125: nan
1164967130: nan
1164967135: 1.00e+02
Different result again. Part of the problem seems to be that
N and $(date +%s) aren't the same. The first one uses subsecond
precision, the second one does not. So N at 12:34:56 is not really
12:34:56, it is something like 12:34:56.121125479
Anyway, this modified script should produce equal results for all of
us, no matter when run:
sdate=1164981600
echo '* Create rrdnan.rrd'
rrdtool create rrdnan.rrd --step=5 \
--start 1164981600 \
DS:testli:GAUGE:7:0:100 \
RRA:AVERAGE:0.5:1:16
echo '* Feed data'
declare -i i
declare s_date
for i in $(seq 1 10); do
# no need to sleep 5
sdate=$((sdate+5))
if [ ${i} -ne 6 ]; then
rrdtool update rrdnan.rrd --template=testli ${sdate}:100
echo (${i} Updated ${sdate}:100
else
echo (${i} NOP
fi
done
echo '* Fetch data'
rrdtool fetch rrdnan.rrd AVERAGE --start=1164981600 --end=start+60
echo '* Finished'
It still produces two NaNs:
1164981625: 1.00e+02
1164981630: nan
1164981635: nan
1164981640: 1.00e+02
#2 Why are there two NaN's?
I thought it is because RRD always requires two values to build a row:
the current and the previous value. This would mean for the first NaN,
the current is missing and for the second NaN the previous is missing.
But probably I'm totally wrong.
Two values are needed when RRDtool needs start and end of an interval,
and compute a rate from it. This is the case for a COUNTER. However,
for GAUGE, this is not needed.
This said: an unknown interval can be longer than just one interval.
In our current script, the amount of time for the known data at timestamp
1164981635 is unknown, therefore the interval 1164981630..1164981635 is
also unknown. Makes sense after all.
If an explicit NaN update at time 1164981630 would have occured, the
rate 1164981625..1164981630 would have been unknown but last update
time at 1164981630 would have been known. The update at 1164981635
would then have succeeded, producing rate 100 in stead of NaN.
You have simplified the discussion by altering heartbeat. What you
have really done is hide the truth: for timestamps _*exactly*_ equal
to a whole number times step, and with a heartbeat twice as large as
the step size (or more), there is _*no*_ gap:
1164981615: 1.00e+02
1164981620: 1.00e+02
1164981625: 1.00e+02
1164981630: 1.00e+02
1164981635: 1.00e+02
All what's needed is to alter heartbeat 7 back into 10 (read: twice
the step) and stop using subsecond precision.
The update at 1164981635 is not more than ${heartbeat} seconds after
1164981625, thus it is valid, thus its rate is used in stead of NaN.
HTH
--
Alex van den Bogaerdt
http://www.vandenbogaerdt.nl/rrdtool/
--
Unsubscribe mailto:[EMAIL PROTECTED]
Helpmailto:[EMAIL PROTECTED]
Archive http://lists.ee.ethz.ch/rrd-users
WebAdminhttp://lists.ee.ethz.ch/lsg2.cgi
[rrd-users] Re: Compromised value after NaN
On Fri, Dec 01, 2006 at 03:26:00PM +0100, Alex van den Bogaerdt wrote: (side note: 1.1806045000e+02 seems a weird value. I am going to look into this) This seems to be a minor bug, fixed in svn. It has to do with rounding (and subsecond precision timestamps indeed). -- Alex van den Bogaerdt http://www.vandenbogaerdt.nl/rrdtool/ -- Unsubscribe mailto:[EMAIL PROTECTED] Helpmailto:[EMAIL PROTECTED] Archive http://lists.ee.ethz.ch/rrd-users WebAdminhttp://lists.ee.ethz.ch/lsg2.cgi
[rrd-users] Tobi @ LISA'06 and Sponsoring
Good Evening. Some News ... * I will be attending LISA'06. If you want to take the oportunity to meet to discuss ideas, projects, war stories, please drop a line. * In order to be able to spend more time on RRDtool/MRTG/SmokePing work, I have setup a Sponsorship Program for companies and individuals who are interested in seeing these projects continuing to evolve. See the homepages for more detail. http://oss.oetiker.ch/mrtg http://oss.oetiker.ch/rrdtool http://oss.oetiker.ch/smokeping See you at LISA! all the best Tobi -- Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten http://it.oetiker.ch [EMAIL PROTECTED] ++41 62 213 9902 -- Unsubscribe mailto:[EMAIL PROTECTED] Helpmailto:[EMAIL PROTECTED] Archive http://lists.ee.ethz.ch/rrd-users WebAdminhttp://lists.ee.ethz.ch/lsg2.cgi
