On Monday, May 5, 2003, at 05:07 AM, Martin Knoblauch wrote:
Hi,
today I upgraded one of our clusters from 2.5.1 to 2.5.3 (gmond,
gmetad and the web-frontend). Since then the log-files on the gmetad
node get filled with stuff like:
headnode /usr/sbin/gmetad[22664]: RRD_update: illegal attempt to update
using time 1052131942 when last update time is 1052132440 (mini
mum one second step)
I think I know what this is. We recently changed the RRD update logic
to use CLUSTER LOCALTIME
as the rrd timestamp. This was done in 2.5.3 I believe.
Now if your gmetad has a data source which is another gmetad (port
8651), it will try to update its rrds multiple times with the same
CLUSTER LOCALTIME. Why? Because gmetad only updates its XML every
20-30s.
So it is possible for your gmetad to attempt to update its rrds twice
with the same LOCALTIME timestamp, causing the errors you see in your
logs.
This is one of those hard-to-anticipate bugs which occur from
unintended side effects to the system. To fix it, I believe we need to
use the true localtime when updating rrds for which we are not the
"authority" on. (The authority mode is off whenever we get our data
from another gmetad).
I will think about this some more, and make the changes to the new
gmetad in CVS.
-FDS
Also, the graphs for anything are flat. The actuals from gmond look OK
though.
Cheers
Martin
=====
------------------------------------------------------
Martin Knoblauch
email: [EMAIL PROTECTED] or [EMAIL PROTECTED]
www: http://www.knobisoft.de
-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
Ganglia-developers mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ganglia-developers
Federico
Rocks Cluster Group, SDSC, San Diego, CA
