Re: [gpsd-dev] refclock 28 gone wacky on me
Yo Mike! ` On Sun, 19 Jun 2016 15:59:16 -0400 Mike wrote: > On 06/10/2016 03:17 PM, Gary E. Miller wrote: > > Yo Eric! > > > > On Fri, 10 Jun 2016 03:49:22 -0400 > > "Eric S. Raymond" wrote: > > > >> That's... very weird. I've never seen it happen. > > But it would explain some old complaints. > > > > Now that we know which Skytraq option it is, I recall always > > selecting the workaround. > > > >> It sounds as though for some crazy reason the GPS is delivering the > >> PPS pulse late. > > Not late, as such. Late in the second, but the time stamp of the > > fix (xxX.940) is pretty close to when actually sent. > > > >> I'm hard put to imagine how gpsmon could screw this up. > > Or Hal with his O-scope. > > > > RGDS > > GARY > I'm seeing this happen now, PPS is good NMEA is off nearly half a > sec... NMEA delivery viewed in gpsmon is 0.011 after top of second. > > *SHM(1) .PPS.0 l 10 16 3770.000 -0.002 0.004 > -SHM(0) .GPS.0 l9 16 3770.000 -496.59 6.661 Looking good. Now add a 0.500 fudge to your ntp.conf. Like this: server 127.127.28.0 fudge 127.127.28.0 time1 0.500 refid GPS What do your other chimers in 'ntpq -p show'? Youu still have not confirmmed which PPS edge you are on. The leading or trailing. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpX5PvN9vzuV.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: [gpsd-dev] refclock 28 gone wacky on me
Yo Mike! On Sun, 19 Jun 2016 17:46:48 -0400 Mike wrote: > > What do your other chimers in 'ntpq -p show'? Youu still have not > > confirmmed which PPS edge you are on. The leading or trailing. > I'll likely put an arrow through this module before I'm able to get a > scope hooked up too it! So leading or trailing is a coin toss at > this point. Which is why I asked what your other chimers say. If you have a chimer without 30 milliSec of you that is usuually good enough to check your edge. > At that point > NMEA deliver was really late, as in > .940 that I was seeing > earlier. To disabiguate 'late' I assume you mean the NMEA timestamp fractional seconds is for later in tthe second, not zero. > I wasn't real pleased and figured I'd have to reset the > setting to get the NMEA delivery back to top of second. Left is > alone for several hours, come back and gpsmon shows that I was back > to getting delivery close to the top of second again. Yes, that is expected. You need to tetll the Skytrazzq to force the top of the second, and save to flash. > At that point PPS looks good, NMEA is way out, where before I had > powered the module down is was looking pretty good. I'm not sure what you mean by 'way out'. > I'll probably > have to pull that info from the statistics if it's really relevant. I > get that I can fudge out the difference shown above. One shouldn't > have to do that every time something is powered off though. You should be within +/- 150 milliSec if you start with the right fudge. The NMEA not being at the top of the second should not affect that at all. > I believe that the ntpshmmon output was showing that PPS is picking > up the first seen and NMEA is picking up the second. Your ntpshmmon it misleading you. It is corrypted by a bug, that is now fixed. Rerun with the ntpshmmon which is now in git hear. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgprzpxBudwLr.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: [gpsd-dev] refclock 28 gone wacky on me
Yo Mike! On Sun, 19 Jun 2016 18:45:45 -0400 Mike wrote: > > Which is why I asked what your other chimers say. > +199.102.46.75 ( .GPS.1 u 61 64 377 39.468 6.354 2.236 > +fairy.mattnordh 192.5.41.40 2 u 37 64 377 60.559 3.875 0.986 > -origin.towfowi. 204.9.54.119 2 u 26 64 377 77.933 1.970 2.503 You are without all of those to better than 6 milliSec. If you have the wrong edge you would be off 20 milliSec or more. So that is reall good. > >> I wasn't real pleased and figured I'd have to reset the > >> setting to get the NMEA delivery back to top of second. Left is > >> alone for several hours, come back and gpsmon shows that I was back > >> to getting delivery close to the top of second again. > > Yes, that is expected. You need to tetll the Skytrazzq to force the > > top of the second, and save to flash. > Did that which is why I didn't understand the delivery coming at near > the end of the second. It appears thought that the firmware *slowly* > brings the delivery back to where it should be. And only one cycle of NMEA per second? If it takes time to get back to the top of the second I would report that to Skytraq as a bug. > >> At that point PPS looks good, NMEA is way out, where before I had > >> powered the module down is was looking pretty good. > > I'm not sure what you mean by 'way out'. > Saying the NMEA offset was within at least a few milli seconds. Now > I'm nearly .5 second off. Odd. The NMEA offset will not stay closer than +/- 100 milliSec over time. But 500 milliSec is too much. Even if the NMEA wanders around in the second, the offset of the fix from the time of fix should be Try setting the serial speed to 38400 and see if it persists. > Okay, ntpshmmon: version 3.17~dev (revision release-3.16-359-g88190a1) That looks good to me. The NMEA delay and offset not bad. Look good to you? > This just confuses me more... Looks very good to me. What looks off to you? NTP1 is offset under 1 millSec, NTP0 offset about 150 milliSec. Fine numbers. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpJdHdpBikXk.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: [gpsd-dev] refclock 28 gone wacky on me
Yo Hal!
On Sun, 19 Jun 2016 17:24:56 -0700
Hal Murray wrote:
> g...@rellim.com said:
> > Yes, that is expected. You need to tetll the Skytrazzq to force
> > the top of the second, and save to flash.
>
> What does that mean?
The Skytraq GPS, and some others, have two modes they can report
the GPS mode.
The first way, the way we expect, is the GPS computes a new PVT
solution at the top of every second, and then sends a message
that that new fix.
So a fix report would look like this:
{"class":"TPV","device":"/dev/ttyS0","mode":3,"time":"2016-06-20T02:14:34.000Z","ept":0.005,"lat":44.068348315,"lon":-121.313259926,"alt":1176.971,"epx":15.315,"epy":18.206,"epv":43.081,"track":121.0306,"speed":1.582,"climb":0.303,"eps":36.41,"epc":86.16}
Notice the fractional seconds of the fix is .000
The second way is for the GPS to report the fix when it is most convenient
for the GPS cpu to report it. The GPS will emit the fix with fractional
seconds to note the exact time the fix was valid.
{"class":"TPV","device":"/dev/ttyS0","mode":3,"time":"2016-06-20T02:14:34.940Z","ept":0.005,"lat":44.068348315,"lon":-121.313259926,"alt":1176.971,"epx":15.315,"epy":18.206,"epv":43.081,"track":121.0306,"speed":1.582,"climb":0.303,"eps":36.41,"epc":86.16}
Noteice the fix time has the fractional seconds of .940
The problem comes in that gpsd never expected the PVT solution to
come that late in the second. The full sentence may actually be decoded
after the start of the next second, and after the PPS for the next
second is received.
There is likely a not too bad solution, but it is not currently implmented
in the gpsd code.
> bellyac...@gmail.com said:
> > Did that which is why I didn't understand the delivery coming at
> > near the end of the second. It appears thought that the firmware
> > *slowly* brings the delivery back to where it should be.
>
> I've lost track of the details of this discussion.
It happesn, too many similar discussions at the same time. Which is
why I encourage people to recap all the details in each message.
> Many GPS chips seem to have a 100 ms timer that drifts slowly so the
> offset within the second when the NMEA strings come out will slowly
> drift then jump back and start over.
I doubt it is not an intentional timer. It seems to correlate with the
number of sats in the solution. The more sats, the more math to do
before the solution is fully computed.
RGDS
GARY
---
Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703
g...@rellim.com Tel:+1 541 382 8588
pgpcU0aHSGNPT.pgp
Description: OpenPGP digital signature
___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel
Re: The new refclock directive is implemented and documented
Yo Eric! On Mon, 27 Jun 2016 00:21:45 -0400 "Eric S. Raymond" wrote: > The new refclock directive is implemented and documented. Cool. > There will be a *limited* open period for bikeshedding about the > driver names. Since you opened the door... > |shm| T | Shared Memory Driver > |gpsd | T | GPSD NG client protocol I think that leads to confusion since they both, at least for now, come from gpsd. And until the gpsd_json driver has sane, documented, defaults, I fear that people will try it first with gpsd. Maybe: gpsd_shm, and gpsd_json. Or: shm and json. There could be other gpsd variants in the future. RGDS GARY ----------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgp3HHUTPPkrt.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: The new refclock directive is implemented and documented
Yo Eric! On Mon, 27 Jun 2016 05:16:15 -0400 "Eric S. Raymond" wrote: > > Maybe: gpsd_shm, and gpsd_json. Or: shm and json. > There is in theory a better case for calling "gpsd" > something else, but in practice I think JSON is sufficiently flexible > that we'll never have a variant that's truly incompatible from ntpd's > point of view. So, let's call it the json driver. RGDS GARY ----------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpkhhRlNQZo6.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Use of pool servers reveals unacceptable crash rate in async DNS
Yo Eric! On Tue, 28 Jun 2016 19:26:39 -0400 "Eric S. Raymond" wrote: > (You should camp on #ntpsec. Also join our Signal channel - because > that's secured, most of the vuln discussions happen there.) Ah, how do we joing the Signal channel? RGDS GARY ----------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpQXsYUMZaBI.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Use of pool servers reveals unacceptable crash rate in async DNS
Yo Eric! On Tue, 28 Jun 2016 19:47:14 -0400 "Eric S. Raymond" wrote: > Gary E. Miller : > > Yo Eric! > > > > On Tue, 28 Jun 2016 19:26:39 -0400 > > "Eric S. Raymond" wrote: > > > > > (You should camp on #ntpsec. Also join our Signal channel - > > > because that's secured, most of the vuln discussions happen > > > there.) > > > > Ah, how do we joing the Signal channel? > > Install Signal on your smartphone and/or Chrome instance. One of us > can and will add you. Done. Tied to my cell phone: 541-390-3793. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpdPdpyb5JHH.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: adns is looking plausible
Yo Eric! On Wed, 29 Jun 2016 08:19:45 -0400 "Eric S. Raymond" wrote: > I haven't looked at the code itself yet, but from reading the C > header file and the website, adns is looking like a plausible > replacement for our homebrew async-DNS. Gentoo has been slowly moving from adns to c-ares. For example wireshark on Gentoo no longer uses adns, but uses c-ares. https://bugs.gentoo.org/show_bug.cgi?id=513982 Gentoo curl also uses c-ares. I could not find any Gentoo packages that still use adns, it used to be common. There is confusion in that Gentoo co-opted the 'adns' keyword form libadns to c-ares. c-ares is MIT licensed and is here: http://c-ares.haxx.se/ "C89 compatibility, MIT licensed, builds for and runs on POSIX, Windows, Netware, Android and many more operating systems." RGDS GARY ------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpnK7nQp9jNH.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Kernel PPS processing
Yo Hal! On Wed, 29 Jun 2016 03:21:40 -0700 Hal Murray wrote: > So I pulled over the sources and built a kernel with NO_HZ turned off > and NTP_PPS turned on. I'm interested. > The next project is to figure out why it works so much better, or > rather why the normal ntpd can't do a lot better. Can you quantify the better? I would have expected identical... RGDS GARY ----------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgp1NqzBfBAmF.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Kernel PPS processing
Yo Matthew! On Wed, 29 Jun 2016 14:55:23 -0400 Matthew Selsky wrote: > On Wed, Jun 29, 2016 at 11:48:08AM -0700, Hal Murray wrote: > > > Can you quantify the better? I would have expected identical... > > > > Did you look at the graph? > > http://users.megapathdsl.net/~hmurray/ntpsec/glypnod-pps-kernel.png Yeah, your 'No kernel' was aweful. If that is your baseline then you got something really, really, wrong. The scale on the 'kernel PLL' was too small to really tell, but I think I get the same on my RasPi's. Scroll down this: https://pi3.rellim.com/ Look at the 'offset of PPS'. It shows usually better than +/- 15 microSec. Looks like PPS is usually close to a few microSec, but frequent 20 microSec dealsy. I'm not sure what the daily spike is. That is KPPS on git head of ntpsec and gpsd. > We tested booting with "nohz=off intel_idle.max_cstate=0" and it made > a difference in our production clocks. But not kernel PPS? I can see why the max_cstate might help, but if the KPPS is interrupt driven I'm not sure why the nohz would help. Can you quantify the individual effects? And is that kernel PPS, KPPS in ntpsec, or just PPS in ntpsec? RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpQn8HWaOm5A.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Kernel PPS processing
Yo Hal! On Wed, 29 Jun 2016 11:48:08 -0700 Hal Murray wrote: > I'm not sure why you would expect performance to be identical. Because thhey use the same kernel generated time stamp and PLL algorithm. > Dave > Mills and crew went to a lot of effort to get code into various > kernels, including writing a RFC. I'd be very surprised if it wasn't > a significant improvement. I've seen a lot projects do work gone with no gain. You do not know until you try. Why would you expect it to be better? syscal latency should not be an issue sinec the time stamping and adjtime() is in the kernel both ways. RGDS GARY ------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpSrjeZwZv7i.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Kernel PPS processing
Yo Hal! > > > Did you look at the graph? > > > http://users.megapathdsl.net/~hmurray/ntpsec/glypnod-pps-kernel.png > > Yeah, your 'No kernel' was aweful. If that is your baseline then you > got something really, really, wrong. The scale on the 'kernel PLL' > was too small to really tell, but I think I get the same on my > RasPi's. I took another look, and realized I misunderstood the y axis. And that you are plotting loopstats and I'm looking at offsets. So not the bad I thought. To get apples and apples, can you send me your gnuplot formula? I'll add it to my chrony graph. The noise around the main signal on your two graphed lines is not that different. And your 'no kernel' shows the sidereal day oscillation that I expect. From that point of view your 'Kernel PPS' looks wrong. Unless you have a very expensive time-keeping saw-tooth corrected GPS you should see the sideral day osccillation. Maybe what is going on is that the damping in the 'Kernel PPS' is a lot less than in the 'no kernel' case. We know that ntpd takes a longtime to adjust the system clock. From that point of view I prefer your 'no kernel' data. Also, what gpsd and ntpsec versions are you on? Are you even using gpsd? gpsd is making some compromises on noise rejecction. And has changed the compromise in different versions. For fast slewing, gpsd allows any PPS from +/- 100 milliSec from expected. That is because chronyd can be slewing the clock almost 90 milliSec a Sec and we do not want to lose PPS during slewing. Most of the time the system clock is stable, and some people patch their gpsd to a 1 milliSec window. That removes the noise spikes from the PPS offset I see on RasPi. Until the startup glitch in NTPsec is fixed, and maybe even after, I feel gpsd should pass on most of the data it gets and let ntpd deal with it. At some point a dynamic error gate may be in order. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgp0DJaaXVA5j.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Kernel PPS processing
Yo Matthew! On Wed, 29 Jun 2016 16:38:40 -0400 Matthew Selsky wrote: > > Can you quantify the individual effects? And is that kernel PPS, > > KPPS in ntpsec, or just PPS in ntpsec? > > We're using a GPS PCIe card with OCXO HQ with a daemon that writes to > SHM and then ntpd reads from the SHM driver. Very interesting. The list has been wondering if there were SHM drivers other than gpsd. I'm sure Eric would like to document what you are using. Can you be more specific? Like is your SHM driver open source? > Measuring on this server via ntpq -p we were seeing offsets of +/- > 3us without either of these two kernel parameters. With nohz=off, > the offsets were +/- 1us. With both kernel parameters we see offsets > too small for ntpq -p to measure. Very interesting. Many people will reboot tonight. :-) ntpd does keep better stats than ntpq -p shows. Look in the ntpd log files. BTW, are you seeing the sideral day oscillation? > This was all measured while running ntpd Classic. The PLL code has not changed, yet. RGDS GARY ----------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpKKeMtc4mDM.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Kernel PPS processing
Yo Matthew! On Wed, 29 Jun 2016 16:38:40 -0400 Matthew Selsky wrote: > Measuring on this server via ntpq -p we were seeing offsets of +/- > 3us without either of these two kernel parameters. With nohz=off, > the offsets were +/- 1us. With both kernel parameters we see > offsets too small for ntpq -p to measure. Just rebooted my main Intel chimer with "intel_idle.max_cstate=0". It was already using HZ=100 Wow. I thought something was wrong. My local clock offset (peerstats file) has always been hanging around 100ppm. Stable to ±1ppm so I figured that was normal. After reboot the local clock offset started at 9ppm and has been slowyly going down, now under 2ppm. I made note of these setting in the gpsd-time-service-howto.txt Anyone got a guess what the equivalent RasPi setting to turn off power saving would be? RGDS GARY ----------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpIdnRRNC8TV.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Kernel PPS processing
Yo Hal! On Wed, 29 Jun 2016 15:28:29 -0700 Hal Murray wrote: > g...@rellim.com said: > >> I'm not sure why you would expect performance to be identical. > > Because thhey use the same kernel generated time stamp and PLL > > algorithm. > > There are two chunks of PPS code in the kernel with separate RFCs. > One is getting the time stamp. The other is doing the PLL. The > in-kernel PLL is totally different from anything in ntpd. Interesting. So they start with the same time stamp, but PLL differently. I wonder why the PLL's is so different? I'll bet the kernell converence is much faster than the ntpsec convergence. The best answer is likely in between. RGDS GARY ------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgp3YFsjnuNz6.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Kernel PPS processing
Yo Hal! On Wed, 29 Jun 2016 21:35:49 -0700 Hal Murray wrote: > g...@rellim.com said: > > Wow. I thought something was wrong. My local clock offset > > (peerstats file) has always been hanging around 100ppm. Stable to > > ±1ppm so I figured that was normal. > > > After reboot the local clock offset started at 9ppm and has been > > slowyly going down, now under 2ppm. > > Your units don't make sense. Yeah, I'm working on getting the descriptions just right on my graphs. Local clock frequency offset, as opposed to local clock time offset. Sorry I did not specify which offset, but I figure the ppm shoulda clued you in. > Offsets would be in units of seconds. I assume you mean some > fraction of a second. A guess would be microseconds. Nope. ntpd has both time offsets and frequency offsets of the local clock. To be real specific, I'm looking at field 3 of the loopstats file. 'man ntp.conf' calls that: frequency offset, with units of PPM. RGDS GARY ----------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpfTFA9wNyry.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Kernel PPS processing
Yo Hal! On Wed, 29 Jun 2016 22:21:41 -0700 Hal Murray wrote: > > Local clock frequency offset, as opposed to local clock time > > offset. > > Most NTP documentation calls that drift. Its magnitude is not very > interesting when discussing quality of time. Changes over time can > be interesting. It's usually much more interesting to look at the > clock offset. Yes, but not all. And certianly not the loopstats doc, which I cited. It would be nice if the ntp doc where more consistent. I agree it is not the most interesting parameter, but any time a small change give almost a 20x improvement in any parameter I'm interested. I'll need to wait a few days to to see how it all shakes out. RGDS GARY ----------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpcz6CoXQh4U.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: My task list
Yo Eric! On Thu, 30 Jun 2016 09:02:52 -0400 "Eric S. Raymond" wrote: > Ideally I'd just write something like > > #if (_XOPEN_SOURCE >= 600) || defined (STD_C_99) > > but the right predefine doesn't seem to exist. I'm still looking. #if __STDC_VERSION__ >= 199901L RGDS GARY ----------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgp1Jz7NKtJR9.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Kernel PPS processing
Yo Hal! On Thu, 30 Jun 2016 02:15:36 -0700 Hal Murray wrote: > g...@rellim.com said: > > I took another look, and realized I misunderstood the y axis. And > > that you are plotting loopstats and I'm looking at offsets. So not > > the bad I thought. > > I can't figure out what that means. It means your graph is not what I thought it was, so ignore my original comments. > I was plotting the offset column from loopstats. Which offset column from loopstats? There are two offset in there. Ah, I see below from column 2, the time offset. > From another handy graph: > > set ylabel "Offset ms" > set y2label "Drift PPM" So, essentially the same as my first graph here, but different scaling: https://rellim.com/graphs/ I added the cstate boot parameter. Made a huge difference, but I'll need a week to get a lot of data. Loopstats Frequency Offset, went from 100 ppm to 0.4 ppm ± 0.2 ppm! Now I can see when the LF goes up the timing suffers. How does yours compare to the current graph? RGDS GARY ----------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpbXUMK9ROUN.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Kernel PPS processing
Yo Eric! On Thu, 30 Jun 2016 05:51:56 -0400 "Eric S. Raymond" wrote: > Gary E. Miller : > > Anyone got a guess what the equivalent RasPi setting to turn off > > power saving would be? > > turbo=1 in /boot/config.txt, I think. See also > > https://www.raspberrypi.org/blog/introducing-turbo-mode-up-to-50-more-performance-for-free/ "We are happy that the combination of only applying turbo when busy, and limiting turbo when the BCM2835’s internal temperature reaches 85°C, means there will be no measurable reduction in the lifetime of your Raspberry Pi." The exact opposite of what I want. I want to force the RasPi to a nice stable frequency, regardless of load or temperature. Not the fastest mode, but the most stable mode. Having the RasPi slow up when it gets hot, thus affecting all the PLL variables is a bad thing. RGDS GARY ------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpVUnyYii8co.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Zero-configuration ntpd
Yo Eric! On Sun, 3 Jul 2016 07:51:49 -0400 "Eric S. Raymond" wrote: > I think ntpd should choose sane defaults and be able to run without > an ntp.conf. Just because that's good hygiene for every program. Yes. > Default servers should be the global NTP pool. You should ask the pool people first. > Default for statistics can be no stats gathering. Agreed. They just grow forever. Ditto ntp.log that should default to the system syslog. Off-topic: ntpd should have a max number of saved logs. ntpd can not use any system log roller until it sanely does a SIGHUP or can start cleanly. > Default restrictions can be none. Suboptimal, but we expect this to > change when and if Daniel's restriction redesign lands. I would suggest ntpd run in client only mode by default. Very few will run ntpd as a real server. > There is currently no default drift file location. This is where > I am not sure of my ground - should there be one? If not, why not? Judging from Gentoo, agreeing on file locations is always a mess. The simplest thing, but unlikely to get majority vote, is to put everything in the statsdir. I'd select /var/log/ntpstats, but once again I doubt there would ever be a consensus location. ntpd has always started up perversely. The driftfile is an initial attempt to fixing that. Chronyd goes further with starting up nicely, ntpd needs to emulate chronyd and improve on chronyd. No driftfile is a step backward. Off-topic: check out the chronyd -r -R and -s options for some startup hacks. I would propose that ntpd option -N be on by default. Possibly also -g should be on by default. RGDS GARY ----------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpKBnVuX1yWm.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Kernel PPS processing
Yo Achim! On Sun, 03 Jul 2016 12:10:44 +0200 Achim Gratz wrote: > Eric S. Raymond writes: > > Gary E. Miller : > >> Anyone got a guess what the equivalent RasPi setting to turn off > >> power saving would be? > > > > turbo=1 in /boot/config.txt, I think. See also > > No. That used to be "force_turbo=1", but is not needed anymore. As of what kernel? I notice RasPi's have all sorts of weird kernel versions and patchsets. My Gentoo RasPi is at 4.4.14-v7, but wheezy is at 4.1.19-v7. Worse my Odroid is at 3.10! > You > simply define the maximum (and perhaps minimum) frequencies in > /boot/config.txt and chose the proper CPUfreq governor. Do I need to do this, or is 'performance' enough? What would you put in config.txt to set the performance governor? > Default is > ondemand, you will either want powersave (to keep at the low > frequency) or performance (force maximum frequency). So far I > haven't seen a difference between those when it comes to NTP > performance, but I'm running two isolated DCF77 clocks only at the > moment. How are you measuring? Eyeballing 'ntpq -p' or running a week of gnuplots? > I can't find any obvious benefit > from using nohz=off on the kernel line either, but then I'm not yet > using a GPS receiver for the PPS. Seemed to help about 5% for me. I need a few hours of data to see the difference. A good test takes days, but eventually I'll loop back and retest things. My biggest problem is that restarting ntpd puts huge spikes into my timing data, making it hard to see small changes. > Keep in mind that the PPS > timestamping is actually done by the VC4 and not the ARM in the BCM > SoC. Very interesting. That is a new twist. Got a citation for that? RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpv7XccxEPcG.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Kernel PPS processing
Yo Achim! On Sun, 03 Jul 2016 23:20:10 +0200 Achim Gratz wrote: > Gary E. Miller writes: > >> No. That used to be "force_turbo=1", but is not needed anymore. > > > > As of what kernel? I notice RasPi's have all sorts of weird kernel > > versions and patchsets. > > I've not kept notes, but the first Raspbian installation from about > two years ago already didn't need it (until you wanted to void the > warranty by overvolting also). Thanks. > > Worse my Odroid is at 3.10! > > Leave those out of the discussion, they aren't RasPi. They run the same Gentoo as RasPi, just an older kernel. Not sure why. > > Do I need to do this, or is 'performance' enough? What would you > > put in config.txt to set the performance governor? > > If you haven't done any setup for lower and higher frequencies, the > governor will never change frequencies (since there is only one to > chose from). When I had 'ondemand' set (the default), and nothing else set, my RasPi it would toggle between a high and a low frequency. And that took some time to settle. > > How are you measuring? Eyeballing 'ntpq -p' or running a week of > > gnuplots? > > Looking at the distribution of PPS arrival times and the resulting > time offset in the loopstats. You'll need some automated tools to see 5% changes. > As long as there's no thunderstorm around it'll keep time > within +-3ms range for the raw offsets taken every 16 seconds and > +-300µs averaged over 3.5 minutes. A GPS HAT will beat that by over 100x. If that matters to you. Plug and play. > Based on your time plots I'm probably not able to see that small an > improvement. I was trying out the nohz=off in hope to get rid of PPS > spikes that are alway 100ms or 200ms late, I see something like that, just not as big a spike. Turned out to be cron jobs. I'm now nice-ing my cron jobs. > >> Keep in mind that the PPS > >> timestamping is actually done by the VC4 and not the ARM in the BCM > >> SoC. > > > > Very interesting. That is a new twist. Got a citation for that? > > The BCM SoC actually is a media processor with an ARM subsystem and > not the other way around as typically found elsewhere. The timestamp > counter and all I/O is provided by the VC4. Got a citation for that? RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpTGtZUIBQby.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Zero-configuration ntpd
Yo Hal! On Sun, 03 Jul 2016 14:06:24 -0700 Hal Murray wrote: > g...@rellim.com said: > > Default for statistics can be no stats gathering. > > Agreed. They just grow forever. Ditto ntp.log that should default > > to the system syslog. > > The main log file does default to syslog Yup. I assume we both agree that is best? > > Off-topic: ntpd should have a max number of saved logs. > > The default is no log files. I don't think ntpd should get involved > with deleting anything. If nothing else, it's an insecurity > opportunity. Debian has a cron job to do it. (I have to kill it > since I want them saved forever.) Interessting. Gentoo has no such thing. I don't see how rotating its own logs is a problem, but I can go with the paranoia and agree to a cron job. If so, I'd like to see a standard cron job in the official tarball. Trusting the user, or distro, to get it right is also a security risk. Maybe even some config files for common log rollers, like logger. In providing same we also warn people not to do the obvious and just restart ntpd with their log roller, which does bad things... RGDS GARY ------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgptVgdCuDv5n.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: question about upgrading from Classic to NTPsec (packaging issue)
Yo John! On Mon, 4 Jul 2016 11:00:54 -0400 "John D. Bell" wrote: > Do we want to install NTPsec in the same hierarchy as Classic? Or in > an alternative location? (/usr/local? /opt??) The default for NTP Classic is /usr/local/. Keep that. Only distros should put things in /usr/ OTOH, any RedHat RPM should likely over-write Classic. RGDS GARY ------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgp9I279ki9oF.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Kernel PPS processing
Yo Achim! On Mon, 4 Jul 2016 08:43:40 + (UTC) Achim Gratz wrote: > Gary E. Miller rellim.com> writes: > > > The BCM SoC actually is a media processor with an ARM subsystem > > > and not the other way around as typically found elsewhere. The > > > timestamp counter and all I/O is provided by the VC4. > > > > Got a citation for that? > > [MIND THE LINEBREAKS] > https://www.raspberrypi.org/documentation/ > hardware/raspberrypi/bcm2835/BCM2835-ARM-Peripherals.pdf Good stuff, but I find no mention of GPIO timestamping.. > https://www.raspberrypi.org/documentation/ > hardware/raspberrypi/bcm2836/QA7_rev3.4.pdf Nothing at all about GPIO or timestamping. > http://www.broadcom.com/docs/support/ > videocore/VideoCoreIV-AG100-R.pdf Nothing at all about GPIO or timestamping. Did I miss something? > There seems to be nothing up yet for the BCM2837 (Pi3), but it's > basically just replacing the A7 cluster in the BCM2836 with an A8 > cluster. Yeah, I'm wondering why the dealy in Linux kernel for 64 bit A8? RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpDUSmxpiCh6.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Kernel PPS processing
Yo Achim! On Tue, 05 Jul 2016 07:35:40 +0200 Achim Gratz wrote: > Gary E. Miller writes: > > Nothing at all about GPIO or timestamping. > > > > Did I miss something? > > Maybe the fact that all the GPIO registers and the timestamp counter > are on the VC4 side of the SoC. The Linux kernel just reads those > when it gets interrupted by the VC4, the ARM subsystem doesn't have > any connections of its own to the outside. I looked for any GPIO timestamp counter in those docs. I could not find it. Got a page number? Or maybe see where the gpio driver reads such a thing? > On another tangent back to NTP, I'm wondering if it wouldn't make > sense to offload the timestamp filtering at least to the VC4. Most > NTP boxes would run headless anyway, so there'd be 16 processors > sitting idle for that sort of thing. Gack. It would be non-portable and very host specific. Once the timestamp is put on the PPS pulse by the kernel driver there is no time critical task left to do, and the PPS thread is pretty small. Maybe if once gcc gets a good generic GPU offloading mode, but I suspect the context switching would swamp any benefits. > >> There seems to be nothing up yet for the BCM2837 (Pi3), but it's > >> basically just replacing the A7 cluster in the BCM2836 with an A8 > >> cluster. > > > > Yeah, I'm wondering why the dealy in Linux kernel for 64 bit A8? > > It wouldn't buy anyone anything of immediate use except having a > complete additional distro to build I use Gentoo. Gentoo is a source distro, so I'm rebuilding it on every host anyway. > and maintain and more memory > pressure to deal with. Yes, memory usage is worse on a 64 bit processor. On the flip side, it makes it a lot easier for the kernel to deal with RAM over 2 GB, so maybe Pi's would get more RAM. The big thing for NTP and gpsd would be the 64 bit math. Both do a lot of 64 bit math. Going to native A53 mode from A7 would also buy us better floating point, better SIMD, more GP registers, crypto extensions and security extensions. And maybe even hope for big.LITTLE mode, which would likely not help the time keeping application > I suspect that eventually a 64bit port will be > added anyway to tick a checkbox somewhere. Linux is trying to kill off x86 (32bit) mode. So I expect 32 bit will get less love in the future. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgp5kIk3h0lZu.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Kernel PPS processing
Yo Hal! On Tue, 05 Jul 2016 12:19:18 -0700 Hal Murray wrote: > g...@rellim.com said: > > The big thing for NTP and gpsd would be the 64 bit math. Both do a > > lot of 64 bit math. > > You can do 64 bit arithmetic without using 64 bit pointers. Yes, Linux on Intel can have a 64 bit kernel, and run 32 bit binaries. But it only just recently implemented. I suspect ARM would also need a 64 bit kernel to support 32 bit pointers and 64 bit arithmetic. > Somebody mentioned that the plan is have one boot file that runs on > all Raspberry Pis. And more CPUs than just Pi's. The trick is to get all the magic constants into the device tree files. Not done yet, but in progress. Linux on Intel can also run a lesat common denominator kernel and then load the needed modules for extra features. Gentoo shows that compiling for the current CPU can speed things up nicely, so I always go custom kernel. Just going from loadable modules to built-in make a nice speed difference. > Are things setup so that user code builds that > way too? Nope. The NEON and SIMD differ between ARM CPUs. And other details like the Trustwave stuff, etc. So most ARM code is either least common denominator, or for those lucky enough to have a source distro: compiled for the current CPU. So my Gentoo get better performance on the Pi3 than Debian. If so, it might take a magic option to the compiler to get > it to use the 64 bit instructions. Plus a 64 bit kernel. A 32 bit kernel does not know how to save and restore 64 bit registers, or even know how many registers there are. Linux never figured out 64 bit binaries on a 32 bit kernel, although it was tried. Unlikely ARM ever will. RGDS GARY ----------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpJ8iJkaMGTW.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: question about upgrading from Classic to NTPsec (packaging issue)
Yo John! On Tue, 5 Jul 2016 16:00:39 -0400 "John D. Bell" wrote: > The executables ended up in /usr/sbin (except for ntpstat, which > ended up in /usr/bin). Drift file in /var/lib/ntp/drift; statistics > in /var/log/ntpstats. Main config file in /etc/ntp.conf, other > bits'n'bobs of config in other places in /etc. :P ISTR that if you > download the Classic source code from ntp.org, configure it with the > defaults, then build and install it, the results do end up in > the /usr/local hierarchy. What a mess! But a well documented mess. That is basically consistent with the FHD (Filesystem Hierarchy Standard). Same old thing as the FSSTD, they just changed the name. https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard Some eople get grumpy if you fail to follow the standard. > I agree with Eric's previous recommendation to *not* install in a > different place (to prevent error reports from a "mixed" install), > but I'm leaning towards having NTPsec's initial RPM file put things > into /usr/local, Sort of conflicting. Only distros are allowed to install into /usr. User installed stuff is supposed to go in /usr/local/ So it the package come from NTPSec, it has to go in /usr/local. But when a distro repackages it they have to put it in /usr/ > and using the /etc/alternatives symlink trick to > pick out the 'right' version. And of course having an uninstall > script which backs it all out, and undoes the symlink indirection. /etc/alternatives is totally non-standard. Better to replace the old versions in /usr with links to /usr/local/. Or, maybe instead a script to hunt out older ntpd's. A lot of programs do this, you find out when your install is taking forever and the installer is searching all your NFS shares. :-( RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpLUMagA4gsz.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Kernel PPS processing
Yo Achim! On Tue, 05 Jul 2016 21:58:12 +0200 Achim Gratz wrote: > Gary E. Miller writes: > > I looked for any GPIO timestamp counter in those docs. I could not > > find it. Got a page number? Or maybe see where the gpio driver > > reads such a thing? > > There is no hardware timestamping, the GPIO just raise interrupts. > But there is one single system time counter on the VC4 side (STC) > that is used to create all timestamps (even those on the ARM side), > advancing each microsecond (it could be run at different speeds, but > 1MHz it is when running Linux). Well, early it was asserted that such a thing did exist. I'm glad that is now put to rest. > > Gack. It would be non-portable and very host specific. Once the > > timestamp is put on the PPS pulse by the kernel driver there > > is no time critical task left to do, and the PPS thread is pretty > > small. > > The idea would be to timestamp at the VC4 side entirely, then use a > new PPS kernel module to read the results back into ARM for ntpd to > use. Some ARM kernel guru would have to do that. NTPsec is portable code. > > Maybe if once gcc gets a good generic GPU offloading mode, but I > > suspect the context switching would swamp any benefits. > > You'd cut out the context switches and the interrupt latency from the > critical timing path. Reduce, not eliminate. The VC4 has its own context switching and latency. > Plus there's a 19.2MHz clock on that side of > the SoC that could potentially be used to get better resolution > timestamps. Which can provdie 38.4MHz resolution. But we are still trying to get the simple GPIO fix to read both PPS edges in the kernel. I would enjoy watching someone try to work this idea in. > > On the flip side, it makes it a lot easier for the kernel to deal > > with RAM over 2 GB, so maybe Pi's would get more RAM. > > Sorry to burst your bubble, but the VC4 is architecturally limited to > exactly one single GB of RAM. Who mentioned VC4? That is the video part of the BCM2837. We were talking about the linux kernel on ARM. When you add swap the virtual memory size goes way past 1GB. And as I said, I expected this to lead to more RAM, I did not say it already had that much RAM. > > The big thing for NTP and gpsd would be the 64 bit math. Both do a > > lot of 64 bit math. > > You can already do that, the length of the addresses doesn't have > anything to do with that. Citation please? I know that addresses and math are only slightly related, but even the Intel can't do 64 bit math with a 32 bit kernel. Unless something changed recently. When doing a context switch a 32 bit kernel does not know how to save 64 bit registers. I've watched people try this on Intel and fail. > > Going to native A53 mode from A7 would also buy us better floating > > point, better SIMD, more GP registers, crypto extensions and > > security extensions. > > You can already target neon. The crypto and security extensions may > not even be implemented. I'm more interested in the 64 bit math for NTPsec and gpsd. That alone would be worth it for me. > > And maybe even hope for big.LITTLE mode, which would likely > > not help the time keeping application > > Not implemented either, both the BCM2836 and the BCM2837 have a > symmetric quad-core cluster. Sad... > > Linux is trying to kill off x86 (32bit) mode. So I expect 32 bit > > will get less love in the future. > > That's an entirely different kettle of fish, mainly going away because > such hardware is hard to come by these days. I don't care why it is going away, but the fact it is going away will mean fewer eyes on the code, thus declining code quality. Same thing happened in the 8 to 16, and 16 to 32 bit tranistioned. The older code bit tatted. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpEAgw04emXb.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Kernel PPS processing
Yo Dan! On Mon, 04 Jul 2016 19:27:50 -0500 Dan Drown wrote: > Quoting "Gary E. Miller" : > > Good stuff, but I find no mention of GPIO timestamping.. > > pps-gpio takes the timestamp in the kernel interrupt handler. Thanks for the confirmation. That was also my understanding. RGDS GARY ----------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpl73JBqPRCm.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: SIGHUP catcher, issue #78
Yo Eric!
On Sun, 10 Jul 2016 07:03:09 -0400
"Eric S. Raymond" wrote:
> Hal Murray :
> > We should get a chance to test the new leap file stuff soon. It's
> > time for a new one. Besides, a day or two ago, the news said they
> > announced one for the end of this year so we'll get to test the run
> > time too.
>
> Gary, we should ship an GPSD release with the expected leapsecond in
> its table sometime soon so we don't have to be in a rush at year end.
I was sorta thinking of September. No one will notice anything
released in the late summer. That work?
> > I think there is a script to fetch a new leap file. This could be
> > added to it.
> >
> > For logrotate on Linux, you can put a fragment like
> > this in /etc/logrotate.d/ntpd
> >
> > /var/log/ntp/ntpd.log {
> > monthly
> > postrotate
> > /usr/bin/killall -HUP ntpd
> > endscript
> > rotate
> > }
>
> Please add that to the distribution etc directory with an explanatory
> comment.
I'm glad to see that happen. Another long standing NTP Classic problem
shot in the head. Once I get caught up from my trip I'll give it a shot.
For a default I'd rotate daily or weekly and only save a month or
two total.
RGDS
GARY
---
Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703
g...@rellim.com Tel:+1 541 382 8588
pgpOAx4PnoQXz.pgp
Description: OpenPGP digital signature
___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel
✘ADEV of GPS Serial and PPS timing
Yo All!
With a lot of help from the time-nuts, I have an Allan Deviation plot
of GPS Serial and PPS time:
https://rellim.com/graphs/adev.png
That is for 6 weeks of ntpd peerstats data.
Now I just gotta figure out what it means. :-)
Eventually this should get into whatever visualizations we come up with.
For some reason this is what hard core time-nuts always want to see.
The hard part, computing the ADEV, is from Tom Van Baak's adev5 program.
Source is here: http://www.leapsecond.com/tools/adev5.c
The script to run it is trivial:
fgrep 127.127.28.0 /var/log/ntpstats/peerstats* > tmp1.log
sort -n < tmp1.log > tmp2.log
awk '{print $5}' < tmp2.log > tmp3.log
adev5 /a 0 6 .30103 < tmp3.log > adev-gps.log
fgrep 127.127.28.1 /var/log/ntpstats/peerstats* > tmp1.log
sort -n < tmp1.log > tmp2.log
awk '{print $5}' < tmp2.log > tmp3.log
adev5 /a 0 6 .30103 < tmp3.log > adev-pps.log
gnuplot < adev.plot > adev.png
As is the plot program:
set terminal png size 900,600
set title "Frequency Stability"
set title font "Arial,28"
set xlabel "Sampling interval, τ (seconds)"
set xlabel font "Arial,24"
set ylabel "Allan Deviation, σ(τ)"
set ylabel font "Arial,24"
set logscale
set format x "10^{%L}"
set format y "10^{%L}"
set zero 1e-20
set tics nomirror scale 2
set mxtics 10
set mytics 10
set style line 1 lw 2 pt 7 ps 1.2 lt rgb "blue"
set style line 2 lw 2 pt 7 ps 1.0 lt rgb "red"
set grid xtics ls 1 lt rgb "gray80" lw 1
set grid ytics ls 1 lt rgb "gray80" lw 1
set datafile separator whitespace
set key top right box width 3
plot "adev-gps.log" using 2:5 title 'GPS' with linespoints
linestyle 1, \ "adev-pps.log" using 2:5 title 'PPS' with
linespoints linestyle 2
I'm guessing a Python person could recode in Python pretty quickly?
Any takers?
RGDS
GARY
---
Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703
g...@rellim.com Tel:+1 541 382 8588
pgpNu5c_ryErr.pgp
Description: OpenPGP digital signature
___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel
✘adev.py
Yo All!
Please don't laugh too hard. Attached is my first Python program, to
make the adev plot.
Next I need to figure out Python process control and get rid of the
shell script.
Still unresolved, the best way to handle minpoll andmissing data points.
RGDS
GARY
---
Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703
g...@rellim.com Tel:+1 541 382 8588
adev.plot
Description: Binary data
#!/usr/bin/env python
# this program takes an NTP peerstats time and changes it to
# a unix time stamp. The rest of the line is unchanged.
#
# a peerstats line looks like this:
# 57589 0.805 127.127.28.1 961a -0.21843 0.0 0.000233609 0.05153
#
# column one is the GPS week, column two is the seconds into the week.
#
# our result will be:
#1468972800.805 127.127.28.1 961a -0.21843 0.0 0.000233609 0.05153
#
# for regular expressions
import re
# for stdin, stdout
import sys
if 2 != len(sys.argv):
print 'Usage:\n' \
' timestamps.py [refclock]\n'
exit(1)
# I found compiling the regex was no win
# so don't do it
# tuples for empty refclock lines
refs = []
for line in sys.stdin:
#print line
# week time refclock offset etc.
m = re.match( "^([0-9]+) ([0-9.]+) ([0-9.]+) ([^ ]+) ([^ ]+) (.*)", line)
if None == m:
# bad line
continue
if sys.argv[1] != m.groups()[2]:
# wrong refclock
continue
# calculate the UNIX time
mjd= long( m.groups()[0] )
second = float( m.groups()[1] )
epoch = 24 * 60 * 60 * mjd + second - 3506716800L;
#print '{:-.3f}'.format(epoch) , m.groups()[2] , m.groups()[3]
refs.append( (epoch , m.groups()[2] , m.groups()[4]) )
# sort by time
refs.sort( key=lambda ref: ref[0])
for ref in refs:
print ref[2]
adev.sh
Description: application/shellscript
pgplIIwFmsmSJ.pgp
Description: OpenPGP digital signature
___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel
Re: adev.py
Yo devel@ntpsec.org! On Wed, 20 Jul 2016 16:35:15 -0700 Hal Murray , devel@ntpsec.org wrote: > g...@rellim.com said: > > # a peerstats line looks like this: > > # 57589 0.805 127.127.28.1 961a -0.21843 0.0 > > 0.000233609 0.= 05153 # column one is the GPS week, column two > > is the seconds into the week. > > Where did you get that? The insides of ntpd know nothing about GPS. > > The ntpd stats files use MJD and seconds of day. Sorry, misleading comment. The variable is indeed named mjd in the code. > This is backwards, but you can reverse it: Uh, oh. I copied that part directly from Dan Drown's timstamps perl in his chrony-graph package. > MJD_1970 = 40587 # MJD for 1 Jan 1970 > > now = time.time() > day = int(now)/86400 > sec = now - day*86400 > mjday = day + MJD_1970 Are you saying the unix time stamp result in the output is wrong? RGDS GARY ----------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpNp4J1T4FzY.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Removing the worst cruft
Yo Eric! On Sat, 23 Jul 2016 15:32:56 -0400 (EDT) e...@thyrsus.com (Eric S. Raymond) wrote: > But AUSTRON/IRIG/CHU...I think there's a good (though not absolutely > dispositive) case for simply dropping them all. I'll give you two out of three. I think IRIG is a really important one. It is the primary time source in high end audio and video work. RGDS GARY ------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpSuz9GZRPlK.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Removing the worst cruft
Yo Mark! On Sun, 24 Jul 2016 01:30:05 + Mark Atwood wrote: > Re IRIG being "primary time source in high end audio and video work" > > You are kidding me. SIgh. Sadly, not. > Can you expand on that for me please? My son does a lot of robotics, and some of the robots he works on are now doing high end work holding high speed cameras filming TV commercials and movies. NDA prevents me from saying the clients and technically I should not even know. But you would know some of them very well. Prolly by Xmas time you will be seeing some amazing new photography styles. The audio, video and robotic equipment is all syncronized to IRIG time. This becomes very important when you get up to 1,000 frame a second stuff. I was also very surprised that IRIG was alive and well in this niche. I propsoed to them using NTP to sync the robots to the IRIG signal, but for now they are slaving the robot on an IRIG time sync box, and running everything else synce on IRIG. RGDS GARY ----------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgp4rnC7N4W6Z.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Removing the worst cruft
Yo Mark! On Sun, 24 Jul 2016 02:52:00 + Mark Atwood wrote: > I know that IRIG is still live, but... is this particular application > feeding the IRIG into a NTPD? Not yet, but that was what I quoted. I believe it is done at other places. Here is a commmercial IRIG to NTTP converter: http://www.buenoptic.net/gps-time-servers/item/330-irig-b-to-ntp-converter And another: http://www.ese-web.com/irig.htm Maybe too small a niche for NTP to care about, but not as small a niche as LORAN receivers. I have a LORAN here cheap if anyone want to try thatt. :-) RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgp1Ue5DHDVCl.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Removing the worst cruft
Yo Mark! On Sun, 24 Jul 2016 03:25:14 + Mark Atwood wrote: > I'm going to think hard here. > > Eric, dont pull IRIG out just yet. > > What I am wishing for, would be for someone to write a standalone in > its own demon process IRIG driver, that then speaks GPSD or SHM to > NTPsec. But testing such a beast would be specialized task. Yes, ideally. RGDS GARY ----------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpNdu3XE92B6.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Removing the worst cruft
Yo Eric! On Sat, 23 Jul 2016 23:10:45 -0400 "Eric S. Raymond" wrote: > And, asnyway, I don't think you're describing a requirement for NTP to > *take in* IRIG as a primary clock source, either. Why do that? If the > IRIG sync box has any kind of digital output, even something as simple > as a one-wire 1PPS export, you run any NTP you want to be downstream > of the master sync clock from that rather than the IRIG. I don't know why, or if their are better ways, I'm just saying people do. Anyone in broadcast or film media has IRIG servers they use, and they want time from them. Far more commonly than I imagined until recently. Several commercial NTP products do it, we wantt them to convert from NP Classic to NTPsec. Right? RGDS GARY ----------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpqu_GyVCl7B.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Removing the worst cruft
Yo Eric! On Sun, 24 Jul 2016 00:26:50 -0400 "Eric S. Raymond" wrote: > Gary E. Miller : > > On Sat, 23 Jul 2016 23:10:45 -0400 > > "Eric S. Raymond" wrote: > > > > > And, asnyway, I don't think you're describing a requirement for > > > NTP to *take in* IRIG as a primary clock source, either. Why do > > > that? If the IRIG sync box has any kind of digital output, even > > > something as simple as a one-wire 1PPS export, you run any NTP > > > you want to be downstream of the master sync clock from that > > > rather than the IRIG. > > > > I don't know why, or if their are better ways, I'm just saying > > people do. Anyone in broadcast or film media has IRIG servers they > > use, and they want time from them. Far more commonly than I > > imagined until recently. Several commercial NTP products do it, we > > wantt them to convert from NP Classic to NTPsec. > > Yes. But there's still the issue that the IRIG driver I removed was > designed to handle analog input via an obsolete class of sound card. Yeah, a driver that does that should be shot. Just don't count out IRIG as a source. > 500us > resolution wouldn't be good enough for what they want. Certainly good enough for 120 frames per second. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgp4CwunH_RHK.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Removing the worst cruft
Yo Hal! On Sat, 23 Jul 2016 21:15:34 -0700 Hal Murray wrote: > g...@rellim.com said: > > Several commercial NTP products do it, we wantt them to convert > > from NP Classic to NTPsec. > > Are they sending IRIG or listening to it? A quick google shows me both ways. RGDS GARY ----------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpHCGULaFDJA.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Removing the worst cruft
Yo Eric! On Sun, 24 Jul 2016 07:35:48 -0400 "Eric S. Raymond" wrote: > Sigh...next time something like this comes up, please try to avoid > causing unnecessary panic. Especially the day before I go on vacation. Then don't raise new issues the day before you go on vacation? RGDS GARY ----------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpmDKwGldopL.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Kernel PLL graphs
Yo Hal! On Mon, 01 Aug 2016 02:40:11 -0700 Hal Murray wrote: > Here are the before and after graphs: > http://users.megapathdsl.net/~hmurray/ntpsec/PPS-kernel.png > The data is from two separate days so this isn't a clean comparison. > I don't know what that machine was doing on either day. I would have expected your non-kernel PLL to be 30x or 100x better. Even on a RasPi2 I get much better, -400 ᵤSec to +800ᵤSec offset on the PPS: https://pi2.rellim.com/day/#PPS Are you sure your kernel PPS (RFC 2783) time stamping is working? RGDS GARY ----------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgp99hpntzALq.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
✘drift file
Yo All! Eric asked me to write up why I thought the chrony drift file handling is better than NTPsec's handling. 1. On startup chronyd checks the time stamp on the drift file. if the timestamp > sysclock, the sysclock is set to the timestamp This is a nice sanity check on the system clock. 2. ntpd stores the frequency ppm offset in the driftfile. chronyd stores the frequency ppm offset and the 'skew' (estimated accuracy of the existing frequency value). Knowing the 'skew' at startup allows chrony to better reject bad reclock input. I can see that saving the 'skew' is a nice touch, but I suspect much the good chronyd startup behavior is explained elsewhere. In a related topic, it would be nice (maybe an option) for ntpd to hold off logging the initial aweful data until after the -g option has set the system clock. And a bit longer, so the wonky startup data is masked. RGDS GARY ----------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpXQHiertbIm.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
✘ntpviz
Yo Eric!
No rush on these, but i was playing with stats today, so I looked at
ntpstats. Looking good, but not ready for me to try to use live.
1. not installed by default
2. no help:
# ./ntpviz -h
option -h not recognized
3. not finding my Liberation fonts:
# ./ntpviz -d /var/log/ntpstats
warning: liberation truetype fonts not found
chrony-graph finds them just fine, they are in:
/usr/share/fonts/liberation-fonts/
And I have the Sans-regular:
/usr/share/fonts/liberation-fonts/LiberationSans-Regular.ttf
3. not working options:
# ./ntpviz -d /var/log/ntpstats --clock-offset
option --clock-offset not recognized
# ./ntpviz -d /var/log/ntpstats --clock-jitter
option --clock-jitter not recognized
# ./ntpviz -d /var/log/ntpstats --clock-stability > tmp.plot
option --clock-stability not recognized
4. --peer-jitters, works.
Missing 99%, 90%, etc. lines and values.
Why '--peer-jitters" when it only takes one argment, and can only be used
once?
5. --peer-jitters, works.
Missing 50% line and value. I'd also like the 90% and 5% lines/values.
Why '--peer-offsets" when it only takes one argment, and can only be used
once?
6. error handling needs work:
# ./ntpviz -d /var/log/ntpstats --peer-offsets 204.17.205.8 > tmp.plot
Traceback (most recent call last):
File "./ntpviz", line 107, in
plot = stats.peer_offsets_gnuplot(show_peer_offsets)
File "/u1/src/NTP/ntpsec/ntpstats/ntpstats.py", line 225, in
peer_offsets_gnuplot
return self.peerstats_gnuplot(peerlist, 4, "Peer clock offset")
File "/u1/src/NTP/ntpsec/ntpstats/ntpstats.py", line 221, in
peerstats_gnuplot
stderr.write("No such peer as %s" % key)
NameError: global name 'stderr' is not defined
7. "-n name" seems to do nothing.
8. looks good: -all-peer-offsets
9. looks good: -all-peer-jitters
10. -s broken:
# ./ntpviz -d /var/log/ntpstats --peer-offsets 204.17.205.1 -s 1470021644
> tmp.plot
Traceback (most recent call last):
File "./ntpviz", line 48, in
starttime = iso_to_unix(val)
File "/u1/src/NTP/ntpsec/ntpstats/ntpstats.py", line 282, in iso_to_unix
return calendar.timelocal(time.strptime(tv, "%Y-%m-%dT%H:%M:%S"))
AttributeError: 'module' object has no attribute 'timelocal'
11. "-e endtime" would be nice.
Then I could do a nice week or month plot without having to run
exactly after the period end.
12. seems to do nothing by default:
# ./ntpviz
#
13. not sure what to do about html...
RGDS
GARY
---
Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703
g...@rellim.com Tel:+1 541 382 8588
pgp91DNteS_3A.pgp
Description: OpenPGP digital signature
___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel
Re: ✘drift file
Yo Mark! On Tue, 02 Aug 2016 19:34:16 + Mark Atwood wrote: > If we make this change, framing it as "it's how chronyd has been > doing it for the past N years" makes it a much easier sell. chronyd setting sysclock to driftfile time is only since Agu 2014. In Oct 2015 they made it optiona with their -s option, musta been some problems for some people. With the non-default -s option it is backward compatible and people can use it if they want. A lot of distros, like Gentoo, preceed ntpd with a shell script to set a worst case sysclock from a file touched on shutdown. So the -s is just internalizing common practice. Gentoo calls it /etc/init.d/swclock. > Especially if we can make the file format the same. ntpd puts just the freq ppm offset on one line in the driftfile. chronyd puts the freq ppm and the freq skew on one line in the file. It would be easy to change ntpd to accept the second parameter, and always write the second parameter. Then maybe only use the second parameter with a non-default option. Only if people like it and gain confidence would it then become Best Practice, or even default. > What principled objections would the hardcore time nerds have? We > do have to keep their needs firmly in mind. I can't see how they can object if the two features are non-default. Saving the freq skew for restart could improve startup performance, but how much is TBD. With the option we can test it. RGDS GARY ------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpUychWOiBPE.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ✘drift file
Yo Mark! On Tue, 02 Aug 2016 19:34:16 + Mark Atwood wrote: > If we make this change, framing it as "it's how chronyd has been > doing it for the past N years" makes it a much easier sell. Forgot an important part. chronyd has saved the freq ppm and freq skew in the driftfile since Jan 2006. Over 10 years ago. RGDS GARY ----------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpiTV8Yiv4dP.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: driftMime-Version: 1.0
Yo Hal! On Wed, 03 Aug 2016 03:13:47 -0700 Hal Murray wrote: > g...@rellim.com said: > > 1. On startup chronyd checks the time stamp on the drift file. > > if the timestamp > sysclock, the sysclock is set to the > > timestamp > We have more important things to do. A few lines of code, faster to code than to argue about it. Better yet, steal tthe chronyd code for it. > The OS should be doing that sort of thing, probably using the root > directory. Why stop with the drift file? Should we check the log > files too? We can't trust the OS to do the rightt thing about time. The OS trusts ntpd to do tthe right thing, but it does not. > It's the sort of code that is hard to test and likely to have subtle > problems. Really? Just 'touch /var/log/ntppstats/driftfile' to a time in the future, start nttpd, and then check the system time. > I think it's a good item to put on the what-do-customers-want list. I assure you Rasi uuers really want this. The lack of an RTC causes real problems. Every time I reboot a RasPi I curse ntpd's poor startup behavior. Almost enough to go back to chronyd. > > 2. ntpd stores the frequency ppm offset in the driftfile. > > chronyd stores the frequency ppm offset and the 'skew' > > (estimated accuracy of the existing frequency value). > > > I can see that saving the 'skew' is a nice touch, but I suspect > > much the good chronyd startup behavior is explained elsewhere. > > I'm not sure that ntpd has a parameter equivalent to skew. I think this is what ntpd calls 'RMS Jitter'. I'm not clear on the ntpd internals, but I do know it has methods for clock selection, and they fail badly on startup. ntpd does not need to exactly mirror what chronyd does. But it clearly needs better start up behavior, and since chronyd starts up muuch better than ntpd that is a good place to start looking. > Again, I vote that we don't do anything now. The current startup > stuff is broken. There is no point in working on things like this > until we understand and fix the current problems. Clearly chronyd understands the problem, so looking at that code is ppart of the path to unuderstanding and fixing the problem. > g...@rellim.com said: > > In a related topic, it would be nice (maybe an option) for ntpd to > > hold off logging the initial aweful data until after the -g option > > has set the system clock. And a bit longer, so the wonky startup > > data is masked. > > But that is when you really really want the logging. Sometimes, but then it takes a week for my graphs to be readable again... > I might agree to put it someplace other than the normal place. Works for me, or maybe a switch. Or maybe fix the startup problems. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpYVgQOhWGDO.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Kernel PLL graphs
Yo Matthew! On Wed, 3 Aug 2016 14:49:44 -0400 Matthew Selsky wrote: > I'm using maxpoll of 1 on my stratum 1 servers. And I have !NO_HZ > set. My offsets stay belong 1 microsecond as reported by ntpq. If > we switched the units to nanoseconds, that might be interesting. chronyc reports to the nanoSec. ntpd pputs nanoSec in the peerstats log file. > I don't have !NO_HZ set on my stratum 2 servers, but I'm looking at > the ramifications of that. I found no huge difference, but a reduuction in the odd wild hare measurement. Noticeable on chrony-graph. RGDS GARY ----------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpNK2efABe_L.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
✘build with Python 3.4 broken.
Yo All! I just tried to compile ntpsec on a new gentoo install. It failed. pi ntpsec # ../Do-config-ntpsec ../Do-config-ntpsec: line 3: cd: ntpsec: No such file or directory Waf: The wscript in '/usr/local/src/NTP/ntpsec' is unreadable Traceback (most recent call last): File "/usr/local/src/NTP/ntpsec/.waf3-1.8.20-c859ca7dc3693011756f4edf45c36626/waflib/Scripting.py", line 104, in waf_entry_point set_main_module(os.path.normpath(os.path.join(Context.run_dir,Context.WSCRIPT_FILE))) File "/usr/local/src/NTP/ntpsec/.waf3-1.8.20-c859ca7dc3693011756f4edf45c36626/waflib/Scripting.py", line 129, in set_main_module Context.g_module=Context.load_module(file_path) File "/usr/local/src/NTP/ntpsec/.waf3-1.8.20-c859ca7dc3693011756f4edf45c36626/waflib/Context.py", line 354, in load_module try:exec(compile(code,path,'exec'),module.__dict__) File "/usr/local/src/NTP/ntpsec/wscript", line 11, in from pylib.configure import cmd_configure File "/usr/local/src/NTP/ntpsec/pylib/configure.py", line 154 print "IDDescription" ^ SyntaxError: Missing parentheses in call to 'print' Waf: The wscript in '/usr/local/src/NTP/ntpsec' is unreadable Traceback (most recent call last): File "/usr/local/src/NTP/ntpsec/.waf3-1.8.20-c859ca7dc3693011756f4edf45c36626/waflib/Scripting.py", line 104, in waf_entry_point set_main_module(os.path.normpath(os.path.join(Context.run_dir,Context.WSCRIPT_FILE))) File "/usr/local/src/NTP/ntpsec/.waf3-1.8.20-c859ca7dc3693011756f4edf45c36626/waflib/Scripting.py", line 129, in set_main_module Context.g_module=Context.load_module(file_path) File "/usr/local/src/NTP/ntpsec/.waf3-1.8.20-c859ca7dc3693011756f4edf45c36626/waflib/Context.py", line 354, in load_module try:exec(compile(code,path,'exec'),module.__dict__) File "/usr/local/src/NTP/ntpsec/wscript", line 11, in from pylib.configure import cmd_configure File "/usr/local/src/NTP/ntpsec/pylib/configure.py", line 154 print "IDDescription" ^ SyntaxError: Missing parentheses in call to 'print' Ah, Gentoo now uses Python 3.4 by default. Switching back to 2.7 and things are better. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgp7IboHNbAvj.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: driftMime-Version: 1.0
Yo Hal! On Wed, 03 Aug 2016 15:15:42 -0700 Hal Murray wrote: > g...@rellim.com said: > > A few lines of code, faster to code than to argue about it. Better > > yet, steal tthe chronyd code for it. > > I could make a similar argument. In this case, I think it doesn't > apply. > > I can kludge up a one time test. We can't test all the strange > cases. Yes, the risk is low, but we need to focus on important > things. Anything that might improve startup will be a big win. Saving a couple more state variables could be that win. > We need to get Eric working on TESTFRAME. I think it's long past > time to stop getting distracted by things like this. I already spend > too much time sorting out minor bugs introduced by changes. Yes, Eric needs to focus on TESTFRAME, once he finishes the ntpviz. The ntpviz at least tells you the build is working. Something lacking now. chrony-graph has picked up a lot of subtle things for me. RGDS GARY ------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgp_KPC72V8t2.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: driftMime-Version: 1.0
Yo Hal! On Wed, 03 Aug 2016 15:54:19 -0700 Hal Murray wrote: > g...@rellim.com said: > > Yes, Eric needs to focus on TESTFRAME, once he finishes the ntpviz. > > The ntpviz at least tells you the build is working. Something > > lacking now. chrony-graph has picked up a lot of subtle things for > > me. > > I'd put ntpviz behind TESTFRAME. We'll have plenty of time to work > on visualization while collecting interesting test cases to feed to > TESTFRAME. Except ntpviz only took a few days, and likely only needs another day or two. Just needs some tweaks to the graphs, and a harness to create all the graphs automagically from the ntp.conf. Plus maybe an html generator. Bsaically the hard part is copied from chrony-graph, and a bit more to do. TESTFRAME might take the rest of the year. RGDS GARY ----------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpTyl2gSfKuf.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: driftMime-Version: 1.0
Yo Hal! On Wed, 03 Aug 2016 16:23:44 -0700 Hal Murray wrote: > g...@rellim.com said: > > Except ntpviz only took a few days, and likely only needs another > > day or two. Just needs some tweaks to the graphs, and a harness to > > create all the graphs automagically from the ntp.conf. Plus maybe > > an html generator. > > Plus all the other stuff that you toss in once that gets working well > enough to see what else you want. I've been using chrony-graph for months now. When ntpviz mostly duplicates that, I'll be happy for the mear term. > > TESTFRAME might take the rest of the year. > > It will take even longer if Eric keeps working on other things. > > We need to get Eric focused on TESTFRAME. I suspect that Eric has not dived into TESTFRAME since he does not have a good mental model he likes yet. That is the hard part, and the part I think he is unsure on. If it were me, I'd consider shimming a few syscalls: adjtime(), send(), recv(), clock_gettime(). Maybe at the runtime linker level, maybe at the libntpd_lib.a level. Then run a good ntpd in record mode. With that saved data, a test ntpd could be run in playback mode. To keep the playback time small, maybe also a way to save internal state to a file, and reload that state in a test ntpd. A nice state dump could also be a good debugging tool. RGDS GARY ----------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpedT0ahv_tP.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Next point release
Yo Mark! On Thu, 11 Aug 2016 21:28:56 + Mark Atwood wrote: > It is time for another point release. Not yet, unless something is urgent. In the middle of August no one will see it. It will get lost in the haze of summer. People are out and about, or watching the Olympics. If you wait until most Universities are back in session then all the script kiddies will jump on it. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpy44H5Z3NgY.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Possible abuse from fetching the leap second file
Yo Hal! On Mon, 15 Aug 2016 14:09:07 -0700 Hal Murray wrote: > k...@roeckx.be said: > > You don't need to "update things". Just point ntp.conf at that > > file. With update things I was just thinking that I would need to > > add a trigger when tzdata is update so I can either reload the > > config file or restart ntpd, depending on what's needed to get it > > to check the file. > > If you send SIGHUP to ntpd it will reload a new file. You can see > that in syslog or the ntpd log file. I just pulled git. I tried -HUP, this is all that happened: 08-15T14:39:18 ntpd[19189]: Saw SIGHUP 08-15T14:39:18 ntpd[19189]: reopen_logfile: same length, ignored What I need ntpd to do on a -HUP is reread ntp.conf. > SIGHUP will kill ntp classic so doing nothing seems like the > conservative approach. I think it's good enough. Yes. RGDS GARY ----------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpyIAKVBE4EH.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Wish list - hack to monitor a server
Yo Hal! On Sun, 21 Aug 2016 22:36:06 -0700 Hal Murray wrote: > I'm thinking of something that will monitor a NTP server and send > mail/SMS/whatever when it finds troubles. I use Icinga2 (nagios fork) for that. NTPsec prolly just needs to document that existing usage. > I can see two types of trouble. One is when the server isn't > responding. The other is that one or more of the upstream servers or > refclocks it is using stops responding. I can see us including a nagio/icinga2 plugin for that, but the existing plugins have worked great for me for over a decade. Here are the existing plugins: /usr/lib64/nagios/plugins/check_ntp_peer /usr/lib64/nagios/plugins/check_ntp_time > There are lots of possible variations/options/parameters. Yeah. Here are the existing options. spidey ntpsec # /usr/lib64/nagios/plugins/check_ntp_peer Usage: check_ntp_peer -H [-4|-6] [-w ] [-c ] [-W ] [-C ] [-j ] [-k ] [-v verbose] spidey ntpsec # /usr/lib64/nagios/plugins/check_ntp_time Usage: check_ntp_time -H [-4|-6] [-w ] [-c ] [-v verbose] [-o ] /usr/lib64/nagios/plugins/check_ntp_time > This should probably wait until Eric gets the python library ready > for use under ntpq. I mention it now because I thought of it and/or > it might provide input to the requirements for that library. Seems to me that's reinventing what is already a very sturday, well trusted and well deployed wheel. New bug: https://gitlab.com/NTPsec/ntpsec/issues/101 RGDS GARY ----------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpWOy7RLFI_2.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
✘ntpviz on missing ntpstats, bug #102
Yo Hal! Check out bug #102: https://gitlab.com/NTPsec/ntpsec/issues/102 I think I fixed your crashes on bad ntpstats directory and missing peerstas and cputemp. Please tests. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpcfaVkQSGYr.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Wish list - hack to monitor a server
Yo Hal! On Mon, 22 Aug 2016 20:34:41 -0700 Hal Murray wrote: > g...@rellim.com said: > > I use Icinga2 (nagios fork) for that. NTPsec prolly just needs to > > document that existing usage. > > I think it's more complicated than that. > > Somebody needs to look at their source code and see what sort of API > they are using. Then we need to document that API so we don't "fix" > something and break their code in the process. No API. You run a commmand line program, it returns: OK, Warning or Error. > (or help them clean things up if they have done something we don't > want to support) I've been using it for over a decade, I can't think of anything I'd change. RGDS GARY ------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgp8FHN8haieJ.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Wish list - hack to monitor a server
Yo Hal! On Mon, 22 Aug 2016 22:19:16 -0700 Hal Murray wrote: > g...@rellim.com said: > > No API. You run a commmand line program, it returns: OK, Warning > > or Error. > > API may not be the right term, but they have to be using some > interface. OK. > Their program talks to our stuff somehow. Sure, it just make an ordinary time request like any other NTP client. Which in the end is what we care about, we want to know that an ordinary client would see a good and ordinary response. RGDS GARY ------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpYmFQWaUeiy.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Comments about ntpviz
Yo Hal! On Tue, 23 Aug 2016 01:53:35 -0700 Hal Murray wrote: > I have a script that uses rsync to collect the log files from various > machines. Eric is working on something like that. > Mostly, I plot 2 days, yesterday and as much of today as is available. So two graphs, one for today and yesterday? ntpviz can do that. > There are more kludgy scripts that split -peer into > -peer- ntpviz does that. > A lot of the graphs I'm interested in need two scales, one that shows > everything and another that ignores the outliers and shows the main > stuff. For those, I add something like: > set yscale [xx:xx]; replot > pause -1 Yeah, I do see that is handy now and then. > I have a few graphs that are long running, months. For those, I > typically use different colors on odd/even days, and another pair of > colors for alternate months. Here are a couple of samples: Looks like a holiday tree, I find it hard to ficus on the red/green pairs. > Some of gnuplots colors are hard to see, at least on my setup with my > eyes. Yellow is the worst case. Yeah, if you have any specificc suggestions on how ntpviz can fix that. But please no red/green barber polls... > If a server is down for a while, the long straight line linking over > that gap looks ugly to me. Yeah, a down server is ugly. It should be shown ugly. > Using microseconds for everything is hard to read for long delays > typical of network links. In bufferbloat cases, I see delays of > several seconds. I've seen delays of 10s of seconds. I can't count > all those 0s quickly. I think milliseconds would be better. (Looks > like Gary tweaked things while I was typing this in.) Yup, I'm working on autoscaling. A lot more to do. I'm thinking of just clipping the outliers ( < 0.5%, > 99.5% ) with a text message of the worst ones > The round trip time is interesting, especially if you have a > bufferbloated DLS link. Yes. Dan Drown had those. They are in the work queue. > If you assume that both your local clock and the server's clock are > accurate, you can get the network transit delay in each direction. > If you plot that you can match steps in the round trip time with > steps in a particular direction. An interessting idea, but NTPsec needs to focus on core ntpd needs until those are nailed down. > There is interesting data in clockstats, at least for some drivers. > One of the important ones is the count for the PPS driver. It's real > interesting if your PPS pulse isn't wide enough to work all the time. Yeah, been looking at that. Since ntpd is undersampling the PPS it can be either good, or real bad. I'm tempted to get Eric to fix the bug first, but maybe I'll need to data so he sees the bug first. > If you have a busy server, there is interesting stuff in usestats. Such as? RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpvdvQv_bS3U.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: PPS undersampling
Yo Eric! On Tue, 23 Aug 2016 19:32:55 -0400 "Eric S. Raymond" wrote: > Hal Murray : > >We > > should get rid of the current every-second timer unless some > > refclock needs it. (battery power) The PPS API has an optional > > wait/wakeup option. We should use that if available. The timer > > stuff also covers when to transmit a packet, but we should peek > > ahead and set the timeout on the select for as long as possible. > > But all that should wait until after TESTFRAME so we can test it. > > Agreed. This has been on my long-term to-do list for some time. Sadly, PPS with the current algorithm should bump up to sampling every one-half second. Some even want 10 Hz or even 100 Hz. Let us leave it on the long-term to-do list and not hash over it now. The subject is complicated and contentious. Way too many moving parts in a core chunk of the code to touch without TESTFRAME. I also expect TESTFRAME will smoke out some of these issues and also have its own different timer loop needs. RGDS GARY ----------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgp2XECZu7CYl.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: PPS undersampling
Yo Hal!
On Tue, 23 Aug 2016 16:13:04 -0700
Hal Murray wrote:
> > Yeah, been looking at that. Since ntpd is undersampling the PPS it
> > can be either good, or real bad. I'm tempted to get Eric to fix
> > the bug first, but maybe I'll need to data so he sees the bug
> > first.
>
> I don't know of any ntpd bug in this area. Do you have a test case?
> Or data from a buggy run?
Not a formal bug, but a bug none the less. Easy to reproduce: reboot
your host, with an intentional off swclock. Then look at how many PPS
sample you get per poll period. At the perverse end ntpd can miss
one in 12 samples.
I have proposed this simple test many times. I have run it myself many
times. Have you done that yet? I have also proposed other simple tests.
For some reason no one ever comes back with data to show my tests show
no problem.
Misunderstandings in this area cause people to break the PPS code in
gpsd every few years. I keep having to revert the changes and show
people the data that says their changes are bad
Having been there, done that, so many time in the PPS code I am not
prepared to take two weeks of my time to explain it again until
Eric is prepared to take 4 weeks of his time to fix it.
> There are/were glitches in some of the gpsd utilities. I fixed one a
> long time ago and I think you fixed one recently. But they are using
> a different mechanism.
Yes, totally unrelated.
> Your mention of Nyquist on IRC a few days ago makes me think that you
> don't understand this area or more likely are just grabbing a word in
> a related area for a similar problem.
Nyquist was a key part of my EE degree. I worked for and consulted to
many test equipment companies for many years. I reverse engineered
a lot of HP gear. So yes, I'll put my knowledge of Nyquist up with
anyone's.
BUT, I do not expect anyone to care about the previous paragraph. When
the timme is right, when NTPsec is prepared to understand and accept
improvements, I will dust off my test cses for one and all to see.
In tthe end the data will speak for itself.
> Nyquist refers to analog signals. If your signal has a bandwidth of
> X Hz, you need 2*X samples per second in order to be able to
> reconstruct the signal.
Mostly correct, except for the word 'analog', and some gross over
simplification.
FWIW, note that Wikipedia agree with me:
https://en.wikipedia.org/wiki/Nyquist%E2%80%93Shannon_sampling_theorem
You would do well to read and undertand that article, as a start.
I actually spent time in the dark ages creating UARTs out of TTL. If
you are up for archeology I suggested you look at how that is done, and
how Nyquist-Shannon theory was such an important part.
> The gpsd code I fixed was doing something like:
> while true {
> if sampleready() then dosomething()
> sleep 1 second
> }
No, that is the gpsd code that broke it. Since fixed.
> The code in ntpd doesn't work that way. It runs off a timer that
> signals and resets itself. That signal goes off every second, the
> same rate as the PPS.
Which is proveably wrong.
> But the ntpd clock is not unsynchronized. It's running at exactly 1
> second per second in the long term.
And the long term is not the problem. The problem is the positively
aweful startup performance of ntpd.
> Actually, any fix should wait for a bigger cleanup in this area.
And in this we 100% agree. When the time comes I have a number of
experiments for one and all to run that show the problems.
> We
> should get rid of the current every-second timer unless some refclock
> needs it.
Yes, but what drivers really need what is a huge discussion. That is
for the long term todo list.
> But all that
> should wait until after TESTFRAME so we can test it.
And on that we also agree, so no further discussion needed until after
TESTFRAME.
BTW, did you do the soldering ron test I asked you to do?
RGDS
GARY
---
Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703
g...@rellim.com Tel:+1 541 382 8588
pgpQ2iu1h4Mof.pgp
Description: OpenPGP digital signature
___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel
Re: PPS undersampling
Yo Eric! On Tue, 23 Aug 2016 20:07:21 -0400 "Eric S. Raymond" wrote: > Gary E. Miller : > > Sadly, PPS with the current algorithm should bump up to sampling > > every one-half second. Some even want 10 Hz or even 100 Hz. > > > > Let us leave it on the long-term to-do list and not hash over it > > now. The subject is complicated and contentious. Way too many > > moving parts in a core chunk of the code to touch without > > TESTFRAME. I also expect TESTFRAME will smoke out some of these > > issues and also have its own different timer loop needs. > > Hm. I think I get it - and you've just added a pretty powerful reason > to eventually pull the refclocks into a separate daemon by telling me > I need to get ntpd itself out of the PPS-watching business entirely > in order to get rid of that timer. Sadly that does not solve the problem. You still have the problem of how to watch the PPS watcher. :-) There will be many ways to skin this cat, but first we need to fatten up this cat, obtain the other ingredients of the stew, and boil the water. So let us leave the fun stuff of ripping up the algorithms until after TESTFRAME. RGDS GARY ------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpRmO_1U_zSI.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Which interfaces are important today to define a time-hardware snap interface
Yo Christian! On Wed, 24 Aug 2016 16:29:31 +0200 Christian Ehrhardt wrote: > I tried to summarize what an "time-hardware" interface for snappy > - Do we also need like /dev/ttyS%d+ ? A lot of procedures on the internet refer to /dev/ttyS%d, /dev/ttyUSB%d, and /dev/ttyAMA%d So for generatlity those would be nice. > To be honest - I personally only ever "met" /dev/pps, but then i'm no > NTP expert so please advise what is would be reasonable to define > such an interface. /dev/pps%d is often autogenerated when needed by gpsd or ldattach. So many NTPP users never know it is there. RGDS GARY ------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpag2PXxlmxC.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: pool vs nopeer
Yo Eric! On Wed, 24 Aug 2016 08:30:01 -0400 "Eric S. Raymond" wrote: > Conclusion: TESTFRAME just went from "we need it someday" to > critical-path. Well, we were preparing for another swing at it anyway. +1. And now that you are getting a gut feel for what ntpd does from the ntpviz and Pi projects, you are ready for it. RGDS GARY ----------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpGxodYeL2jU.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Stratum one autonomy and assumptions about GPS
Yo Eric! On Thu, 25 Aug 2016 00:19:46 -0400 "Eric S. Raymond" wrote: > This was going to be a note to just Hal originally, but it will do the > rest of the team no harm to know more about the scenarios and > assumptions driving some of my design choices. > > Hal objected (off list) to me drawing a conclusion from today's > offset multiplot that check servers aren't necessary when you have > a local GPS - a Stratum 1 really can run autonomously. He said, > correctly of course, that the check servers aren't there to improve > time accuracy when the GPS has sat lock, but to backstop the GPS when > it flakes out. > > I shall now discuss three interlocking reasons this possibility does > not loom as large in my mind as it does in Hal's. > > 1. GPS outage length and frequencies are decreasing Don't care. If you need your NTP to work, you need to know it is working. Otherwise failure are not noticed. > 2. The autonomy scenarios I think about are not hobbyist-budget > productions Yeah, and the big biys REALLY need to know their NTP is right. > 3. There's a lower bound below which outages don't matter; we may be > there. I don't agree. I monitor all my services 24x7, and I do get NTP problems in my logs. > Any given fixed accuracy target for deviation from UTC, combined with > a maximum crystal drift rate, defines a longest tolerable GPS outage. Not the majority failure mode. > We may already be at a technological place where GPS outages don't > bust the tolerable-error budget, even with cheap hardware. If we > aren't, we'll probably be there soon. We can't define a single tolerable error budget. We can provide some ranges of options for the user. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpaaxhDoxlFQ.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Stratum one autonomy and assumptions about GPS
Yo Dan! On Thu, 25 Aug 2016 10:23:50 -0500 Dan Drown wrote: > I had setup a test along these lines a week ago: > https://blog.dan.drown.org/gps-pps-drift-when-it-has-no-signal/ Yeah, and I've been watching it. > My results were a long term average of 250us-500us lost per day > (~3-6ppb). This surprised me because of how low it was, I assume I > got lucky with the long term average temperature of the GPS module. I do not believe your data. After two weeks your GPS is still outputting sat used data. The Ephemeris timed out long ago, and the Almanac almost gone too. I think you still have a weak signal and since you GPS already had a lock it is seeing something. > Of course, this data doesn't apply to all GPS outage situations. Nor does it reflect any test I ever ran. But I admit to never testing high end gear like Trimble gear. RGDS GARY ------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgprg3WERSutl.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Stratum one autonomy and assumptions about GPS
Yo Achim! On Thu, 25 Aug 2016 20:33:36 +0200 Achim Gratz wrote: > Dan Drown writes: > > The GPS module I'm using still outputs PPS even when it loses lock. > > That won't be true for every GPS module. > > I also have a NavSpark mini attached at the moment and I was surprised > it kept outputting the PPS when losing 3D lock (it stops blinking the > LED). Most cheaper GPS always output PPS, even when they have no idea where they are or what time it is. Garmin documents this, as do others. RGDS GARY ----------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpozcThDfHJh.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Stratum one autonomy and assumptions about GPS
Yo Hal! On Thu, 25 Aug 2016 15:30:25 -0700 Hal Murray wrote: > e...@thyrsus.com said: > > I have a USB thermometer on order, they're cheap. Might I suggest > > you get one and repeat this experiment, actually plotting your > > temperature variation? > > Most CPU chips include a temperature sensor. Which I have found does not correlate with any of my NTP data. I now have a lot of data, just need to finish up the ntpviz temp module. RGDS GARY ------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgps41ssZY_xi.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Comments about ntpviz
Yo Hal! On Tue, 23 Aug 2016 13:38:54 -0700 Hal Murray wrote: > The part I forgot to mention is that if I was going to do something > to make my setup better, I would try to make a wrapper to drive > gnuplot - a little window off to the side so I could poke Next rather > than typing return at the gnuplot command window. The main thing I > want is a way to backup a few graphs. If you want to backup your graphs, just add a loggerd script to gzip them to somewhere on a schedule. > The second step would be a > simple way to adjust vertical and/or horizontal scaling (aka zoom > in/out) to cover the cases where I didn't anticipate I would want one > and/or didn't get it right. ntpviz is a cron job thing. No one has come up with a good idea yet. It could just generate a 90% plot and a 100% plot. RGDS GARY ----------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpcLB7yoy_DU.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Stratum one autonomy and assumptions about GPS
Yo Eric! On Tue, 30 Aug 2016 00:08:17 -0400 "Eric S. Raymond" wrote: > Gary E. Miller : > > > 1. GPS outage length and frequencies are decreasing > > > > Don't care. If you need your NTP to work, you need to know it is > > working. Otherwise failure are not noticed. > > OK, the test for "know it is working" is: you have lock, or you had > lock less than x seconds ago where x is a worst-case of your drift > model to whatever confidence interfal you want to fix. Not THE test, A test. Your test would catch few of the failure modes I see. > > I don't agree. I monitor all my services 24x7, and I do get NTP > > problems in my logs. > > And you also said in recent mail that you don't work with the kind of > hardware a serious autonomy-seeker would use. So *your* NTP problems > are not determinative, though they could be useful input data for > improving error-estimation techniques. Correct. There is NO determinative solution, because their is NO determinative use case. I suspect we could probably come up with about 5 use cases, each with VERY different needs. A guy that needs good time to keep his OAUTH/DNSSEC working has very different time stability needs than someone doing audio work, or harder still video work. And if that guy doing OAUTH or DNSSEC just needs it to shop Amazon his uptime needs are very different than someone running a ton of DNSSEC zones. Yet they are all perfect condidates for a GPS/NTP solution. > > Not the majority failure mode. > > That's an interesting statement. What *is*, in your experience, the > dominant failure mode. The GPS goes nuts and start spitting out crazy time. Or no time at all. As previously discussed, there are many causes to get those two effects. > > > We may already be at a technological place where GPS outages don't > > > bust the tolerable-error budget, even with cheap hardware. If we > > > aren't, we'll probably be there soon. > > > > We can't define a single tolerable error budget. We can provide > > some ranges of options for the user. > > And that's exactly what I've been pushing towards - to develop some > statistical modeling on the basis of which we can make estimates to > whatever confidence bound the user wants to set as a parameter. A good friend of mine was a former statistician. He gave it up and became an engineer. He said you only use statistitics until you start to understand the problem. Then other sciences take over. He hated starting to get a handle on the problem, then some specialist would solve it. So, when you fall back on stats, it is because you don't really know what is happening. And yes, clearly we can not even agree on failure modes, so we needs more stats, and that just leads us to a few interesting problesm to colve. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpkRX6JX0fRe.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Comments about ntpviz
Yo Hal! On Tue, 30 Aug 2016 22:56:43 -0700 Hal Murray wrote: > g...@rellim.com said: > > If you want to backup your graphs, just add a loggerd script to > > gzip them to somewhere on a schedule. > > Wrong binding for "backup". > > The context was the way I'm currently looking at graphs. I feed > things like this to gnuplot: > reset; load "foo1.gp" > pause -1 > reset; load "foo2.gp" > pause -1 > reset; load "foo3.gp" > pause -1 > ... I'm noty sure how this appies to ntpviz??? > What I meant by backup was look at the graph I was looking at a few > seconds ago. The above only allows me to go forward to the next > graph. Since the graphs are every 30 mins or hour, I'm not sure what you are trying for? > Some sort of wrapper to drive gnuplot should be simple. It just > hasn't gotten to the top of my list yet. But we already have chrony-graph and ntpviz to frive gnuplot, So I'm confused what you want? RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgp4BAzwIlgc7.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: PPS undersampling
Yo Hal! On Tue, 30 Aug 2016 23:24:26 -0700 Hal Murray wrote: > > Hm. I think I get it - and you've just added a pretty powerful > > reason to eventually pull the refclocks into a separate daemon by > > telling me I need to get ntpd itself out of the PPS-watching > > business entirely in order to get rid of that timer. > > The reason I want to get rid of the every-second timer is to save > power when operating on battery with no refclocks. Saving power is good, but I suspect the extra power is minimal. I hace USB power meters, so we can measure this. > If you have a PPS, you have to take an interrupt every second. No, every 500 millisecond. > I'm > not a wizard on that area. It would be interesting to know if you > can get useful interrupt responses when starting from a power-save > mode. Similarly, can you wakeup in time to get data from a serial > port without dropping any characters? It may be that power-save just > won't happen if you are using refclocks. Easy to test, when we have a knob to twist. It would depend a lot on the mode the CPU is in. I always set governor=performance, so I would see no change in power. > Moving the PPS processing out of ntpd doesn't change any of that. True, whether it is on gpsd or ntpd it should be similar power. RGDS GARY ----------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpvlqTLftZPY.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: PPS undersampling
Yo Hal! On Wed, 31 Aug 2016 01:19:49 -0700 Hal Murray wrote: > g...@rellim.com said: > > Saving power is good, but I suspect the extra power is minimal. I > > hace USB power meters, so we can measure this. > > It depends on the details. You can save a lot of power by turning > stuff off. The more you turn off, the more power you save but the > longer it takes to get running again. I think most CPUs these days > have an instruction that is roughly "halt and wait for an > interrupt". That lets the CPU turn off the clock for the instruction > unit so it doesn't have to burn power running a tight loop. As an > experiment, you could write some code that goes into a tight loop and > see if that takes more power than not running anything. All I care about is how it affects ntpd. As long as ntpd does a usleep() the OS will do the right thing. > The next step is things like putting the memory into self refresh > mode or turning off the disk controller. Turning those off is a deep sleep. That will not work for ntpd. > USB is ugly to power down since there aren't any interrupt wires. Yup, not gonna happen. > In the extreme, you can power off the clock generation circuit. And then there is no point in running ntpd. > Your USB power meter should work for a Raspberry Pi or BeagleBone. I > assume you have a wall-power meter for a laptop or desktop. Yeah, but they all average over a good part of a second. Nothing ntpd does will move the (virtual) needle. RGDS GARY ------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgp8lhTE7PH7a.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Comments about ntpviz
Yo Hal! On Wed, 31 Aug 2016 00:38:58 -0700 Hal Murray wrote: > > I'm noty sure how this appies to ntpviz??? > > I was trying to describe what I'm doing currently and what I like and > don't like about it. > > The current ntpviz makes a bunch of graphs as files and puts a web > page wrapper around it. Yes. > I'd like to avoid that and put the output of > gnuplot directly on the screen. So ignore the html and just look at the png output graphs with the png viewer of your choice. You could multipane, 3D wheel, carosel, etc. Or a simple Python script with some GUI buttons. > I'd also like a bunch of knobs so I can tweak the graph I'm looking > at. Most likely zoom in or out. > I assume all that would be reasonably simple after I knew how to do > it, but it hasn't gotten to the top of my list yet. Pretty much any pnng viewer can do that. Granted the png resolution may be too low, but that is an easy fix. > > Since the graphs are every 30 mins or hour, I'm not sure what you > > are trying for? > > I'm not interested in automatically making graphs for somebody else > to look at. I'm only interested in "now", when I want to look at > things. > > A frequent thing that happens is that I see something interesting on > a graph and then want to look more carefully at a graph I was looking > at a few seconds ago. ntpviz only takes 20 seconds to run on a Pi3 for me, ant then I have all the graphs. Or just ask ntpviz for the one graph you want, prolly about a second, RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgps1T4_QOwkC.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
✘ntpviz ±(rtt/2)
Yo All! I have added ±(rtt/2) to the peer offset charts. I think ntpviz now does pretty much what chrony-graph does. Plus temps, minus sone chronyd stuff. Still on the TODO list: systems stats (LF, etc.) gpsd stats (sats in view, DOP, etc.) Improved formatting Check it out. A live copy is here: https://rellim.com/ntpstats/day/ RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgp6uUzAGA0mx.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: â ntpviz ±(rtt/2)Mime-Version: 1.0
Yo Hal! On Wed, 07 Sep 2016 16:53:06 -0700 Hal Murray wrote: > I prefer RTT rather than RTT/2. > > RTT/2 suggests that the routing is symmetric which is wrong quite > often. . RTT/2 was what Dan Drown used, I'm not sure I care either way. Either way, RTT, or RTT/2, the impression from the graph is of symmetry. So I don't see one is better than the other... RGDS GARY ----------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpXX1W_x2Skc.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: â ntpviz ±(rtt/2)Mime-Version: 1.0
Yo Hal! On Wed, 07 Sep 2016 16:53:06 -0700 Hal Murray wrote: > X-Mailer: exmh version 2.7.2 01/07/2005 with nmh-1.3 Time to get a better email client, yours is mangling my UTF-8 when you respond to me RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpZzh1xFdVoR.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ntpviz (rtt/2)
Yo Dan! Not the first time you have had to explain it. I'll try to summarize this in the index.html. Maybe more can be said on the ntp.conf man page? On Thu, 08 Sep 2016 10:39:07 -0500 Dan Drown wrote: > Quoting Hal Murray : > > I prefer RTT rather than RTT/2. > > > > RTT/2 suggests that the routing is symmetric which is wrong quite > > often. > > Adding +/- RTT/2 on the graph reverses the assumption ntp uses that > the routing is symmetric. ntp already subtracts rtt/2 from the > calculated offset to estimate the one way latency. Removing that > from the equation will show you the relevant offsets of: when the > request started and when the response was received. > > Here's an example of an upstream source that had the request latency > (in green) change due to routing paths: > https://dan.drown.org/vps3/remote-ntpgps.mci.png > > You can see the response latency (in blue) did not change > significantly. So the sudden jump in the NTP offset (in red) was > not due to the clocks changing frequencies relative to each other, > but due to routing effects. > > You can also see that after the 23rd, the new request path had a lot > more jitter that is not in the response path. This also impacts the > offset. > > > ___ > devel mailing list > devel@ntpsec.org > http://lists.ntpsec.org/mailman/listinfo/devel > RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpxveMXmvk2M.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Why does stuff in pylib get compiled twice?
Yo Hal! On Thu, 08 Sep 2016 18:56:30 -0700 Hal Murray wrote: > [111/133] Compiling ntpkeygen/ntpkeygen.c > [112/133] Compiling ntptime/ntptime.c > [113/133] Compiling pylib/__init__.py > [114/133] Compiling pylib/__init__.py > [115/133] Compiling pylib/packet.py > [116/133] Compiling pylib/packet.py > [117/133] Compiling pylib/statfiles.py > [118/133] Compiling pylib/statfiles.py > [119/133] Compiling util/hist.c > [120/133] Compiling util/sht.c https://gitlab.com/NTPsec/ntpsec/issues/110 Yet another waf related bug. Maybe Amar can fix some of these? RGDS GARY ----------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpieFJXSd7iW.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
✘Python broken %
Yo All!
Yet another undocumented Python failure:
i>>> d = {}
>>> d[1] = "one"
>>> d[2] = "two"
>>> str(d[2])
'two'
>>> echo "%s(d[2])s" % locals
File "", line 1
echo "%s(d[2])s" % locals
^
SyntaxError: invalid syntax
Python says %s(x)s works for any x that str(x) works.
Here is the Python doc:
https://docs.python.org/2.7/library/stdtypes.html#string-formatting
"'s'String (converts any Python object using str())."
WRONG!
RGDS
GARY
---
Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703
g...@rellim.com Tel:+1 541 382 8588
pgpGAwwVttFBr.pgp
Description: OpenPGP digital signature
___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel
✘namedtuples overly limited
Yo All!
2nd Python undocumented limitation:
https://docs.python.org/2.7/library/collections.html?highlight=namedtuples#collections.namedtuple
namedtuples makes no mention of a limitation on the number of elements in
a namedtuple.
But:
>>> a = collections.namedtuple('a', 'b', 'c', 'd', 'e')
Traceback (most recent call last):
File "", line 1, in
TypeError: namedtuple() takes at most 4 arguments (5 given)
RGDS
GARY
-----------
Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703
g...@rellim.com Tel:+1 541 382 8588
pgpSoDgZLaZMY.pgp
Description: OpenPGP digital signature
___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel
✘Python 3!
Yo All! Good progress today on getting NTPsec to be Python 3 compatible. Eric got waf 1.9.4 installed. Matt Selsky fixed a huge number of nits that where fatal errors in Python 3. The result is you can now build and install NTPsec with Python 2.7 and Python 3.4! "./waf check" even works. I was not so lucky with ntpviz. Just one annoying thing in ntpviz I can't fix. Documented on line 74. I can make it work in Python 2, or Python 3, but not polyglot... RGDS GARY ------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgppel7KelHPR.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: -g screqup (from IRC)
Yo Hal! On Mon, 12 Sep 2016 13:26:38 -0700 Hal Murray wrote: > > 12:09:40gemiller > > | "killall ntpd; sleep 1; ntpd -N -g" > > | screws up the clock for over a day, > > | much better without the -g > > You have either found a very interesting bug or you are on a wild > goose chase. Intersting, but not new. I've been complaining about btpd startup for a while. -g is just one aspect of it. > The -g switch is supposed to be very simple. It allows a large > initial step. Other than that, it shouldn't do anything. Sadly, simple things have bad consequences. See all my previous posts about nntpd bad startup behavior. > Are you getting steps at all? grep your ntpd.log for "clock_step". > It needs lots of logging turned on. You can test it with the bump > option in ntpfrob. Can you tell me which logging? Then I can test for this case. > steps are reasonably common at first-boot. The size of the step > depends upon how good your TOY/CMOS clock is. RasPis do not have CMOS clocks. So the startup step is huge. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpFow9VGpITI.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: -g screqup (from IRC)
Yo Hal! On Mon, 12 Sep 2016 14:15:05 -0700 Hal Murray wrote: > [to get clock-step logging] > > Can you tell me which logging? Then I can test for this case. > > It's part of: > logconfig =syncall +clockall +peerall +sysall > I think it's in sysall, but I'm not sure. sysall not in the ntp.conf man page. I also do not see anything related to allow_panic that would log when the options is taken: ntp_loopfilter.c > >> The -g switch is supposed to be very simple. It allows a large > >> initial step. Other than that, it shouldn't do anything. > > Sadly, simple things have bad consequences. See all my previous > > posts about nntpd bad startup behavior. > > I'm not saying that there aren't problems, but blaming them on -g > doesn't make sense. > > If -g really is causing troubles, then we need a test case. I got the test case, as in my previous email: # killall ntpd; sleep 1; ntpd -N -g But so many bad things going on there hard to tell one from the other. Looking at my charts you can always tells when I do that, it stands out quite tall. And data to go with my test cases: https://rellim.com/day/ https://pi.rellim.com/day/ https://pi2.rellim.com/day/ https://pi3.rellim.com/day/ I'm trying to get better in the notes, so everyone can see what is going on. Feel free to run your own test cases and port the data. But you are right, I see no way in the code the -g is causing that, so I am getting maybe a false correlation. But clar from above that very bad things do happen on startup. > > RasPis do not have CMOS clocks. So the startup step is huge. > Your recipe was from the command line. That is not the startup case. It is one of my startup cases. For some definition of 'startup'. I can think of many different definitions of startup. Pick one, pick them all. > (unless the ntpd that you are killing never got started, or something > like that) OK, then we can call this case the "stop/start' or 'restart' case. Fair enough? But that is partly why I'm trying to limit myself to smaller cases, like minpoll=maxpoll=N, until those are understood. Way too many random bad things happening at startup to test them all apart right now. Best to focus on the statistically proveable cases first, slow goind, but harder to argue with. I find a minimum test is at least 24 hours, better yet 48 or more. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgp0QcY5tDQ8o.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: -g screqup (from IRC)
Yo Hal!
On Mon, 12 Sep 2016 15:39:26 -0700
Hal Murray wrote:
> The logging stuff goes through a subroutine, one for each of the 4
> types. That subroutine translates a compile time constant to a
> string, maybe logs it, and maybe saves the new state. I'd have to
> check the code to be sure.
>
> If you grep for clock_step, you will find the translation table.
> ./libntp/statestr.c:{ EVNT_CLOCKRESET, "clock_step" },
> If you grep for EVNT_CLOCKRESET you will find the place in
> ntp_loopfilter where it gets used.
Which is only vaguely related to panicgate. So not useful.
> > I got the test case, as in my previous email:
> > # killall ntpd; sleep 1; ntpd -N -g
>
> That test case doesn't show that -g is the problem. I expect you
> would get the same thing without the -g.
Well, duh!
> > But so many bad things going on there hard to tell one from the
> > other.
>
> Then why are you claiming that -g is causing the problems? That's
> why I said "wild goose chase".
Because it does appear to.
> Please start a new thread with a description of the start/restart
> quirks you are seeing and the environment. Graphs and/or test cases
> are good.
You already opened a bug: #45. I'm happy to focus on that one first
as the most obvious to start.
> > I find a minimum test is at least 24 hours, better yet 48 or more.
>
> That doesn't match my experience, at least if you are talking about
> start/restart transients.
All there on my plots. Feel free to post yours.
> I haven't looked carefully.
Please do. That is what I am doing. I don't expect our data to match
that closely. Even my Pi's are all different.
RGDS
GARY
---
Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703
g...@rellim.com Tel:+1 541 382 8588
pgpoTBRr9cvH8.pgp
Description: OpenPGP digital signature
___
devel mailing list
devel@ntpsec.org
http://lists.ntpsec.org/mailman/listinfo/devel
Re: How much do we care about high-load scenarios?
Yo Eric! On Tue, 13 Sep 2016 02:25:41 -0400 "Eric S. Raymond" wrote: > I'd prefer to bias towards architectural simplicity unless and until > field reports force us to optimize for the high-load case. If NTPsec is to seriously challenge NTP Classic, it must outperform NTPsec at the bleeding edge. In the NTP case that is high load factor and high performance. RGDS GARY ----------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpWZte6PrOtr.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
✘minpoll=maxpoll=2
Yo All! I have some solid results on minpoll=maxpoll=1. That gives a poll time of 2 seconds. When minpoll=maxpoll=0 works Ill be testing that. UNtil thne I'll be testing other values. First, the good news. On a Raspberry Pi 3B, with Uputronics hat, minpoll=maxpoll=1 is much better than minpoll=maxpoll=6 (64s). At poll=64: Local Clock Freq Offset: 90% 1.4ppm, Jitter 90% 3.3 us Local Stability 90%: 2.2 ppt PPS Offset 90% 68us, Jitter 90% 9.7 us At poll=2: Local Clock Freq Offset: 90% 1.1ppm, Jitter 90% 0.43 us Local Stability 90%: 2.3 ppt PPS Offset 90% 1.6us, Jitter 90% 0.34 us See attached pi3-local-offset.png. Before 18:25 12 Sep 2016 UTC is poll=64s after is poll=2. Pretty nice, a very good improvement in time, for a slight decrease in frequency. And NTP is all about time, right? Now the bad news. On a quad XEON, with an MR-350P serial GPS: At poll=64 Local Clock Freq Offset: 90% 570ppb, Jitter 90% 10.2 us Local Stability 90%: 1.2 ppt PPS Offset 90% 34.2us, Jitter 90% 16.3 us At poll=2 Local Clock Freq Offset: 90% 495ppb, Jitter 90% 16.9 us Local Stability 90%: 27.7 ppt PPS Offset 90% 39.0us, Jitter 90% 26.7 us See attached spidey-local-offset.png. Before 18:10 12 Sep 2016 UTC is poll=64s after is poll=2. That went really, really badly... So, the answer to the best poll? It depends... RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpp6m3D3Xoqa.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: How much do we care about high-load scenarios?
Yo Eric! On Wed, 14 Sep 2016 16:41:47 -0400 "Eric S. Raymond" wrote: > Eric S. Raymond : > > John D. Bell : > > > *If* I were the architect on this project (thankfully I'm not!) I > > > would get the code simplified and streamlined in single-thread > > > mode as far as possible, and then design and code a > > > POSIX-threading-conformant "improvement". > > > > That is exactly the contingency plan I have unless Higher Authority > > (Mark and LF) tells me we need to do something else and do it now. > > I should amplify that a little. > > I will not willingly take the complexity hit of multithreading > *anything* until I see benchmark numbers demonstrating a bottleneck > that can be effectively addressed in no other way. It's not good > enough to just handwave and say "high load, we gotta optimize" - > without measurements optimization is premature and you know what > *that* means. Sadly, unless you have threading there is no way to A/B test it against no threading. So let us not rip out any existing threading until it gets tested. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgplbPJlBOQic.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: How much do we care about high-load scenarios?
Yo Eric! On Wed, 14 Sep 2016 17:22:51 -0400 "Eric S. Raymond" wrote: > Gary E. Miller : > > Sadly, unless you have threading there is no way to A/B test it > > against no threading. So let us not rip out any existing threading > > until it gets tested. > > There isn't threading any there to rip out, except for the one hack > to avoid stalls on DNS lookups. Different part of the code. Oh, well then it becomes just another feature on the todo list, near the bottom until there is a demonstrated need. I suspect NIST, Apple, M$ and a few others have the problem. If they need a solution they'll ask, and help. RGDS GARY ----------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpNvVvmxhNN3.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ✘minpoll=maxpoll=2
Yo All! I have some solid results on minpoll=maxpoll=2. That gives a poll time of 4 seconds. This results set after about 43 hours of run time. Eric delegated me the task to fix for minnpoll=maxpoll=0. On my list to fix early next week. First, the bbad news. On a Raspberry Pi 3B, with Uputronics hat, minpoll=maxpoll=2 (4s) results are a bit worse than than minpoll=maxpoll=1 (2s). At poll=4s: Local Clock Freq Offset: 90% 1.1ppm, Jitter 90% 0.08 us Local Stability 90%: 2.9 ppt PPS Offset 90% 2.2us, Jitter 90% 0.20 us At poll=2s: Local Clock Freq Offset: 90% 1.1ppm, Jitter 90% 0.43 us Local Stability 90%: 2.3 ppt PPS Offset 90% 1.6us, Jitter 90% 0.34 us At poll=64s: Local Clock Freq Offset: 90% 1.4ppm, Jitter 90% 3.3 us Local Stability 90%: 2.2 ppt PPS Offset 90% 68us, Jitter 90% 9.7 us Poll=2s is still the bast for this hardware, but there is a little tradeoff between offset and jitter. Since NTP is about time, not frequency, we go with the best time. Now the good news. On a quad XEON, with an MR-350P serial GPS: At poll=4s: Local Clock Freq Offset: 90% 332ppb, Jitter 90% 20.9 us Local Stability 90%: 15,6 ppt PPS Offset 90% 40.6us, Jitter 90% 29.7 us At poll=2s Local Clock Freq Offset: 90% 495ppb, Jitter 90% 16.9 us Local Stability 90%: 27.7 ppt PPS Offset 90% 39.0us, Jitter 90% 26.7 us At poll=64s Local Clock Freq Offset: 90% 570ppb, Jitter 90% 10.2 us Local Stability 90%: 1.2 ppt PPS Offset 90% 34.2us, Jitter 90% 16.3 us For this hardware poll=64s is stil the bast, and both low poll times are much worse. Not much to see in the graphs, so I skipped them. Next time (poll=8s) I'll put the numbers in chart form. So, the answer to the best poll? It depends, but poll=4s does not seem optimal for my two cases. So far poll=2s for RasPi+HAT, and poll=64s for Xeon+Serial GPS. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpi8b4JcxS5F.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ✘minpoll=maxpoll=2
Yo Achim! On Fri, 16 Sep 2016 20:17:50 +0200 Achim Gratz wrote: > Gary E. Miller writes: > > Poll=2s is still the bast for this hardware, but there is a little > > tradeoff between offset and jitter. Since NTP is about time, not > > frequency, we go with the best time. > > You 've made this argument before, but I think it's circular > reasoning. Really? I think the data is very clear, you can optimize for time or for frequency. > You make the local clock follow external jitter faster. What external jiitter? Assumption #1 is that the PPS is way better than the internal XTAL. An assumption supported by the GPS data sheets. > Whether or not that gets you closer to true time isn't something you > can decide from your data, since the NTP offset value only tells you > how close the PLL controller thinks it is to the external source. Yes, it is a bit dodgy, using our dependent cvariable to measure itself. I'm open to better ways to measure. But when offset varies by over a factor of 10 it is certainly indicative. Large enough that local peers can see the improvement. And please notice I am optimizing for self-referencetial stable time, not some 'true' time. Until it is stable, no point playing with static offsets to make it 'true'. > > Now the good news. On a quad XEON, with an MR-350P serial GPS: > > I think as that example amply shows (I think you are using serial line > discipline for PPS here), Yes, sort of serial line PPS. The Xeon is RS-232, the RasPi's are actually using the GPIO lines, but very similar results. > having the obviously better (than the rasPi) > local clock follow a noisy reference (likely more jittery than the one > on the rasPi) isn't helping the precision. Uh, lost me. My RasPi and my XEON are following local PPS. That local PPS is far more stable than annything else in the vicinity. Depending on who's talking the PPS may be accurate down to 50 or even 10 ns. No way to tell. So the whole point is to track the PPS as well as possible. When I can track it down to under 1 us then it will be worth comparing different PGS to see the best ones. > BTW, both your plots showed relatively large swings in frequency > offset in a short period of time. Both plots? I'm up to dozens now Gotta be a bit more specific. > Do you have an A/C that is > producing larger temperature swings? Nope. My house has no A/C. It would likely be better if I did have A/C. If you look at the complete graphs (with CPU, Room, HR, etc. temps) you will see a temperature correlation. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpCYTqrOm1_6.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
✘minpoll=maxpoll=3
Yo All! New data for poll=3 (8 Sec). Xeon: Local Clock Local Clock Local ClockPPS PPS Poll Freq Offset 90% Jitter 90% Stability 90% Offset 90% Jitter 90% 1 495ppb 16.9us 27.7ppt 39.0us 26.7us 2 332ppb 20.9us 15.6ppt 40.6us 29.7us 3 165ppb 20.5us 7.5ppt 34.6us 27.9us 6 570ppb 10.2us 1.2ppt 34.2us 16.3us The above is pretty clear that for every 90% measure the best is for poll=6, except Frequency Stability. That is OK since the goal is to optimize for time, not frequency. I think the frequency shifting is due to ntpd pulling the xtal against temp changes. RasPi3: Local Clock Local Clock Local ClockPPS PPS Poll Freq Offset 90% Jitter 90% Stability 90% Offset 90% Jitter 90% 1 1.1ppm 0.43us 2.3ppt 1.6us 0.34us 2 1.1ppm 0.08us 2.9ppt 2.2us 0.20us 3 0.5ppm 0.18us 1.9ppt 3.0us 0.56us 6 1.4ppm 3.3us2.2ppt68us 9.7us Here it is a litle less clear if poll=1 or =2 is best. poll=1 has the least PPS Offset, but for a 30% reduction in offset, the Local Clock Jitter reduces by over a factor of 5. 3 is getting worse and 6 is just bad. I'll probably rerun the =1 and =2 numbers eventually, But poll=4 is running now. More data in a few days. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgp46QuIWUpk2.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ✘minpoll=maxpoll=2
Yo Achim! On Mon, 19 Sep 2016 20:09:04 +0200 Achim Gratz wrote: > Gary E. Miller writes: > >> You 've made this argument before, but I think it's circular > >> reasoning. > > > > Really? I think the data is very clear, you can optimize for time > > or for frequency. > > No, I still think you're misinterpreting it. The only reason you see > that apparent tradeoff is that once you steer the PLL fast enough > (higher loop bandwidth) all the measurement noise ends up in the > frequency variable, while if you steer it more slowly (lower loop > bandwidth), it will be seen predominantly in the offset variable. > These two variables are not independent and are not themselves an > actual measurement of what their names suggest. I'm not sure why you think we disagree. I fully agree with your interpretation. Basically you push here, the error mosve over there, and vice versa. > >> You make the local clock follow external jitter faster. > > > > What external jiitter? Assumption #1 is that the PPS is way > > better than the internal XTAL. An assumption supported by the GPS > > data sheets. > > The PPS may be (at time scales beyond 1s), but not the measurement > that NTP does and uses to steer the PLL. Actually, even at time scales of nanosec, the PPS is very good. But I agree that all we can see is the result of less than perfact time stamping in the kernel, mashed with less than perfect NTP algorithsm. Until we get something better it is what we got. > Looking at the PPS arrival > times on my rasPi (with ppswatch), there is around 10µs jitter when > there is no load and going up to 30µs with light activity. You are being generous. I see much worse sometimes. And you have to specify Pi 1, Pi 2, or Pi 3. I have them all, and they are all different. > I'm > running at poll=4, so ntpd completely eliminates the 10µs jitter in > the loopstats, as it should. This jitter must be assumed to come > from the measurement itself since there's no way the GPS module > produces that much and the local clock doesn't have that large jitter > at the 1s timescale either. I'm running a poll=4 test right now. I'll comment when I have real data. On my Pi 3, the poll=3 is worse then poll=1 and poll=2, so I have my dounts that it gets better at poll=4. But you may have a Pi 2, or a different kernel. If you got data, pleasse send it in. > > And please notice I am optimizing for self-referencetial stable > > time, not some 'true' time. Until it is stable, no point playing > > with static offsets to make it 'true'. > > You are in fact destabilizing it by making it follow spurious > deviations in the measurement. When disciplining an oscillator, you > must not do that for timescales lower than the Allan intercept with > the discipline clock, or you're actually making it worse than it > already is. My data is my data. I look forward to you coming up with better data. If ntpd is calculating 'goodness' wrong then ntpd needs to be fixed. If we can come up with a better metric, even if just for testing and requires hardware, I'd love to hear it. > >> BTW, both your plots showed relatively large swings in frequency > >> offset in a short period of time. > > > > Both plots? I'm up to dozens now Gotta be a bit more specific. > > You've only posted two plots in the post previous to the one I > responded to. The plots are too easy to mis-interpret. The human eye is not good at reading crazy graphs for the 90% point. I dont care so much how the noise looks as long as I have a good 90% band, averaged over 24 hours. I have now posted 5 data measurements for each of 2 very different hosts at 4 different poll values. All 24 hour averages. More to come. I'll report that data after my sig. > > Nope. My house has no A/C. It would likely be better if I did have > > A/C. If you look at the complete graphs (with CPU, Room, HR, etc. > > temps) you will see a temperature correlation. > > Oh, I see it now. You're plotting days-hh:mm, not hh-mm:ss as I > assumed initially. I'm always open to reducing ambiguity, any way the chart could be more obvvious? Also, be sure to look at the Xeon plots. The temps stays within 1°C, but the frequency still swings a lot over the day. I suspect it in more corelated to atmosphere temp then room temp. At 500ppb the small things matter, just not sure which small things. https://rellim.com/ntpstats/day/ RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 Xeon:
Re: ✘minpoll=maxpoll=2
Yo Achim! On Mon, 19 Sep 2016 20:09:04 +0200 Achim Gratz wrote: > Oh, I see it now. You're plotting days-hh:mm, not hh-mm:ss as I > assumed initially. I just pushed a change to the x label. From "Time" to "Time (dd-hh:mm)". Does that resolve the confusion? RGDS GARY ------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpzpK5rI5fbM.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ✘minpoll=maxpoll=4
Yo All! New data for poll=4 (16 Sec). Like before, all data is 90% bounds, run over 24 hours or more. Clearly the proper minpoll=maxpoll is important and non-trivial to find. Xeon: Local Clock Local Clock Local ClockPPS PPS Poll Freq Offset 90% Jitter 90% Stability 90% Offset 90% Jitter 90% 1 495ppb 16.9us 27.7ppt 39.0us 26.7us 2 332ppb 20.9us 15,6ppt 40.6us 29.7us 3 165ppb 20.5us 7,5ppt 34.6us 27.9us 4 465ppb 15.5us 3.1ppt 22.5us 20.1us 6 570ppb 10.2us 1.2ppt 34.2us 16.3us Moving slowly from 1 to 4 seems mostly a good direction. 5 is next. RasPi3: Local Clock Local Clock Local ClockPPS PPS Poll Freq Offset 90% Jitter 90% Stability 90% Offset 90% Jitter 90% 1 1.1 ppm0.43us 2.3 ppt 1.6us 0.34us 2 1.1 ppm0.08us 2.9 ppt 2.2us 0.20us 3 0.5 ppm0.18us 1.9 ppt 3.0us 0.56us 4 1.12ppm1.27us 2.26ppt10.3us 4.11us 6 1.4 ppm3.3 us 2.2 ppt68 us 9.7 us Clearly poll=4 is past the optimum point for this host. More to come. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpbEVmpeQmWV.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: ✘minpoll=maxpoll from 1 to 6
Yo All! New data for poll=5 (32 Sec). Like before, all data is 90% bounds, run over 24 hours or more. Clearly the proper minpoll=maxpoll is important and non-trivial to find. Xeon: Local Clock Local Clock Local ClockPPS PPS Poll Freq Offset 90% Jitter 90% Stability 90% Offset 90% Jitter 90% 1 495ppb 16.9us 27.7ppt 39.0us 26.7us 2 332ppb 20.9us 15,6ppt 40.6us 29.7us 3 165ppb 20.5us 7,5ppt 34.6us 27.9us 4 465ppb 15.5us 3.1ppt 22.5us 20.1us 5 435ppb 13.3us 3.4ppt 17.7us 16.9us 6 570ppb 10.2us 1.2ppt 34.2us 16.3us Moving slowly from 1 to 6 seems mostly a good direction. 7 is next. RasPi3: Local Clock Local Clock Local ClockPPS PPS Poll Freq Offset 90% Jitter 90% Stability 90% Offset 90% Jitter 90% 1 1.1 ppm0.43us 2.3 ppt 1.6us 0.34us 2 1.1 ppm0.08us 2.9 ppt 2.2us 0.20us 3 0.5 ppm0.18us 1.9 ppt 3.0us 0.56us 4 1.12ppm1.27us 2.26ppt10.3us 4.11us 5 1.36ppm1.34us 2.64ppt11.4us 4.43us 6 1.4 ppm3.3us2.2 ppt68us 9.7us Clearly poll >= 4 is past the optimum point for this host. I think the winner is poll=1, but poll=2 or 3 are close. More to come. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpIfrCeBFAis.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
Re: Did your fix for minpoll=0 work?
Yo Hal! On Wed, 21 Sep 2016 10:48:04 -0700 Hal Murray wrote: > I worked out a (much) more complicated fix. I'll happily dump it if > yours works. I have nothing yet. Just a start of a patch, pushed a tiny bit last night. > When I was testing things, ntpq showd a poll of 1, but clockstats was > acting like it was 2. So there are 2 options for what "works" means. Minpoll=0 and maxpoll=0 are used internally to signal NTP_MINDPOLL (6) and NTP_MAXDPOLL (10). So until that is fixed we can not grab 0 to actually mean 0. If you have something that works, love to see it. RGDS GARY ------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 pgpDjRzNVCMSe.pgp Description: OpenPGP digital signature ___ devel mailing list devel@ntpsec.org http://lists.ntpsec.org/mailman/listinfo/devel
