Re: [ntp:questions] Garmin LVC 18x jitter problem
Miroslav Lichvar writes: > I think you do. To prevent the source from being rejected as a > falseticker it needs to have a larger root distance. That could be from > a larger dispersion or delay. You modified the refclock jitter, which > ends up in the peer dispersion (after some weighting). Another approach > would be to specify a minimum for the peer dispersion directly. There > could be also an option to specify a minimum delay. But adjusting it in > the refclock specific code, as you did, makes perfect sense to me. Which of the alternatives do you prefer? I got one way working, but I am willing to test any other in case you have a patch or could at least tell me where to enforce the minimum in the code to create a patch. In the end, it would be great to get the needed functionality into ntpd. Michael ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Garmin LVC 18x jitter problem
Miroslav Lichvar writes: > On 2019-08-12, Michael Haardt wrote: > > I would appreciate if we could focus on the major issues first, like > > why the modified jitter is not shown by ntpq. > > I think the explanation is that you are modifying jitter of individual > samples, but ntpq -p is showing jitter between offsets. Print all > variables of the peer with ntpq -c "rv $ASSID" and look for "jitter" > and "filtdisp". You are right: associd=48733 status=961a conf, reach, sel_sys.peer, 1 event, sys_peer, srcadr=SHM(0), srcport=123, dstadr=127.0.0.1, dstport=123, leap=00, stratum=0, precision=-20, rootdelay=0.000, rootdisp=0.000, refid=GPS, reftime=e0fec9d6.89f0223c Wed, Aug 14 2019 19:56:38.538, rec=e0fec9d7.c9977670 Wed, Aug 14 2019 19:56:39.787, reach=377, unreach=0, hmode=3, pmode=4, hpoll=4, ppoll=4, headway=0, flash=00 ok, keyid=0, ttl=0, offset=-21.688, delay=0.000, dispersion=199.450, jitter=7.766, filtdelay= 0.000.000.000.000.000.000.000.00, filtoffset= -21.69 -24.50 -21.05 -20.86 -24.06 -31.32 -18.34 -39.11, filtdisp=200.00 200.24 200.48 200.72 200.96 201.20 201.44 201.68 Indeed I used minjitter 0.2. I can't say that I understand the difference, though. Do I modify what I should modify? So far I only ever checked the offset and noticed that it does vary in a way that can't be estimated with a (short) filter, and set minjitter in a way that would always include all offsets. It looks like there is a fine difference between jitter and dispersion, and while the code calls it jitter, I modified the dispersion. Could you elaborate? Michael ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Garmin LVC 18x jitter problem
Jakob Bohm writes: > If this is done by setting all bytes of the memory to 0, this doesn't > necessarily map to 0.0 in the floating point formats of all machines. Correct. But it does work for modern systems, which is why the code works. I already mentioned a flag should be used, which avoids the initialization. I would appreciate if we could focus on the major issues first, like why the modified jitter is not shown by ntpq. Michael ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Garmin LVC 18x jitter problem
Jakob Bohm writes: > Minor issue 1: You seem not to initialize fudgeminjitter if not > configured. Probably to 0.0 All values are initialized to 0 when the structs are allocated, I checked that. That's why it works without a flag bit at all. I still like the flag bits more, because they make everything easier to understand. > Wishlist issue 2: Maybe also make fudgeminjitter available for network > clocks, e.g. to describe a non-uniform packet jitter in a network path > or non-uniform hardware jitter in a preferred site local higher stratum > server. Well, I would be rather surprised if the developers accepted the patch as it is due to the described issues, and likely there is a better place in the code to perform this function. But it does show the problem analysis holds and I am quite happy with the result: associd=0 status=0115 leap_none, sync_pps, 1 event, clock_sync, version="ntpd 4.3.99@1.3699-o Wed Aug 7 17:29:04 UTC 2019 (3)", processor="armv6l", system="Linux/4.19.50-1-ARCH", leap=00, stratum=1, precision=-18, rootdelay=0.000, rootdisp=0.344, refid=PPS, reftime=e0fa8018.c9990f56 Sun, Aug 11 2019 13:52:56.787, clock=e0fa8020.9fe6beb8 Sun, Aug 11 2019 13:53:04.624, peer=48734, tc=4, mintc=3, offset=0.000287, frequency=-9.938, sys_jitter=0.003815, clk_jitter=0.004, clk_wander=0.000, tai=37, leapsec=20170101, expire=20191228 Stable operation with these values is not possible without this change. Now the question is how to do implement it in a better way. Michael ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Garmin LVC 18x jitter problem
By now I hacked ntpd for allowing to override the lower jitter bound,
which fixes the problem properly. The attached patch shows the current
state.
Good: It works! ntpd no longer drops the GPS as false ticker, which in
turn does not drop PPS that depends on the preferred peer, so everything
works as beautiful as with "tos mindist 0.1", but without the negative
effect on the root dispersion that is caused by increasing the minimum
distance.
Bad: I don't see the modified jitter with "ntpq -c peers" and I have
no idea why. I verified it is modified. Perhaps there is duplicated
code somewhere I missed to modify as well? I should add a flag, but that
would run out of bits, and I am not certain if I may need to change other
struct members or variables storing flags as well, because much data is
being copied instead of referenced. I would appeciate some help here.
IMHO the minimum distance should only be set in case of multiple
reference clocks that unfortunately have no overlap of offset and jitter,
so increasing the distance allows to use them at the logical price of a
higher root dispersion. Using it to fix the wrong jitter estimation of
a single clock with not statistically distributed jitter is not what it
is intended for. Overriding the jitter of that clock is what is really
required, and checking past mailing lists, this topic came up quite a
few times. Obviously that allows to set mindist to 0, which reduces
the root dispersion to the actual value, which is below 1 ms for me.
Michael
diff -u -r ntp-dev-4.3.99-orig/ntpd/ntp_refclock.c
ntp-dev-4.3.99/ntpd/ntp_refclock.c
--- ntp-dev-4.3.99-orig/ntpd/ntp_refclock.c 2019-06-07 09:42:47.0
+0200
+++ ntp-dev-4.3.99/ntpd/ntp_refclock.c 2019-08-07 19:17:08.947049304 +0200
@@ -494,6 +494,7 @@
}
pp->offset /= m;
pp->jitter = max(SQRT(pp->jitter / m), LOGTOD(sys_precision));
+ pp->jitter = max(pp->jitter, pp->fudgeminjitter);
#ifdef DEBUG
if (debug)
printf(
@@ -1080,6 +1081,7 @@
pp->sloppyclockflag &= ~CLK_FLAG4;
pp->sloppyclockflag |= in->flags & CLK_FLAG4;
}
+ pp->fudgeminjitter = in->fudgeminjitter;
}
/*
diff -u -r ntp-dev-4.3.99-orig/include/ntp_refclock.h
ntp-dev-4.3.99/include/ntp_refclock.h
--- ntp-dev-4.3.99-orig/include/ntp_refclock.h 2016-03-27 23:59:49.0
+0200
+++ ntp-dev-4.3.99/include/ntp_refclock.h 2019-08-07 19:07:13.902963553
+0200
@@ -81,6 +81,7 @@
double fudgetime2; /* configure fudge time2 */
int32 fudgeval1; /* configure fudge value1 */
u_int32 fudgeval2; /* configure fudge value2 */
+ double fudgeminjitter; /* configure fudge minjitter */
u_char currentstatus; /* clock status */
u_char lastevent; /* last exception event */
u_char leap; /* leap bits */
@@ -176,6 +177,7 @@
*/
double fudgetime1; /* fudge time1 */
double fudgetime2; /* fudge time2 */
+ double fudgeminjitter; /* manually set lower bound for jitter */
u_char stratum;/* server stratum */
u_int32 refid; /* reference identifier */
u_char sloppyclockflag; /* fudge flags */
diff -u -r ntp-dev-4.3.99-orig/ntpd/keyword-gen.c
ntp-dev-4.3.99/ntpd/keyword-gen.c
--- ntp-dev-4.3.99-orig/ntpd/keyword-gen.c 2019-06-07 05:54:31.0
+0200
+++ ntp-dev-4.3.99/ntpd/keyword-gen.c 2019-08-06 19:54:05.362952788 +0200
@@ -199,6 +199,7 @@
{ "stratum", T_Stratum, FOLLBY_TOKEN },
{ "time1", T_Time1,FOLLBY_TOKEN },
{ "time2", T_Time2,FOLLBY_TOKEN },
+{ "minjitter", T_Minjitter,FOLLBY_TOKEN },
/* system_option */
{ "auth", T_Auth, FOLLBY_TOKEN },
{ "bclient", T_Bclient, FOLLBY_TOKEN },
diff -u -r ntp-dev-4.3.99-orig/ntpd/ntp_config.c
ntp-dev-4.3.99/ntpd/ntp_config.c
--- ntp-dev-4.3.99-orig/ntpd/ntp_config.c 2019-06-07 10:59:13.0
+0200
+++ ntp-dev-4.3.99/ntpd/ntp_config.c2019-08-07 19:04:12.254768986 +0200
@@ -3703,6 +3703,13 @@
clock_stat.flags &= ~CLK_FLAG4;
break;
+ case T_Minjitter:
+#if 0
+ clock_stat.haveflags |= CLK_HAVEMINJITTER;
+#endif
+ clock_stat.fudgeminjitter = curr_opt->value.d;
+ break;
+
default:
msyslog(LOG_ERR,
"Unexpected fudge flag %s (%d) for %s",
diff -u -r ntp-dev-4.3.99-orig/ntpd/ntp_parser.y
ntp-dev-4.3.99/ntpd/ntp_parser.y
--- ntp-dev-4.3.99-orig/ntpd/ntp_parser.y 2019-06-07 09:42:47.0
+0200
+++ ntp-dev-4.3.99/ntpd/ntp_parser.y2019-08-06
Re: [ntp:questions] Garmin LVC 18x jitter problem
> ntpd was happy with 3.70, I haven't looked at later firmware version. > (Garmin is up to version 4.20 now.) I am using 4.20 due to the GPS week rollover. > The graph shown in http://www.moria.de/~michael/tmp/offset.svg also shows > high drift & jitter, though not as severe as the old > release I used. Yes. It's not pretty, although still usable - if only I could avoid the jitter calculation. I run ntpd with minimum distance set for over a week now and have not seen any problems in the PPS source. I guess I will have to check how hard it is to modify the source for that feature. Michael ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Garmin LVC 18x jitter problem
David Woolley writes: > On 12/07/2019 11:32, William Unruh wrote: > > Do not use nmea as a time source. It it has a long delay ( hundreds of > > ms) which depnds on the speed of connection, on length of the nmea > > sentance, etc. EAch character takes from 100 of usec to ms to be > > delivered, and the lengthof sentences is variable, and the gps sends out > > a sentence only when it is not terribly busy ( which depends on how many > > sattelites it is trying to handle, etc) It is good for setting the > > seconds (sometimes-- if you ask it to send too many sentences it can be > > out by more than a second totally messing up your timing). > > One sentence (RMA) and a high baud rate make it useful for setting the > > second. > > I believe the OP is using it for seconds. The problem is, I think, that > ntpd will only use it, and only accept the PPS, if it is a true chimer. Exactly. I need NMEA for seconds only _and_ it must be accepted as true chimer, otherwise PPS does not work. > Moreover, I think the OP's problem is that the NMEA timing is too > repeatable, so its root dispersion is too small to cover the static > error, although, if they had a lot of jitter, they would then advertise > a root dispersion that doesn't give credit to the PPS accuracy. Almost: The NMEA timing is too repeatable, so its root dispersion is too small for covering the /actual dynamic/ error, so once the jitter center moves, which it does in unpredictable ways, I get a falseticker. The static error is already configured: fudge 127.127.28.0 time1 0.518 refid GPS The time1 was measured by peerstats for some days and agrees with the value documented in other installations. Of course the root cause is awful NMEA jitter behaviour by the device, but I can't change that, and the LVC 18x is already one of the better devices. If the jitter was gauss distributed or equally distributed, ntp would estimate the jitter correctly. The window had to cover a few days as it is, which is not feasible. I would prefer setting a minimum instead of a fixed NMEA jitter to allow ntpd still calculating the jitter value and kicking NMEA out as false ticker in case of malfunction. I just need ntpd not to believe the low NMEA root dispersion it often sees. :) Some examples, just a minute apart: remote refid st t when poll reach delay offset jitter == *SHM(0) .GPS.0 l6 16 3770.000 -17.468 18.600 *SHM(0) .GPS.0 l6 16 3770.000 21.772 20.134 *SHM(0) .GPS.0 l3 16 3770.000 13.032 8.446 In the second and third example, the GPS jitter window around the GPS offset no longer covers the PPS jitter window around the PPS offset, because the GPS jitter value does not react quickly enough. Without setting "tos mindist 0.1", once PPS is used, the root dispersion of my clock is determined by PPS and very low - until the falseticker problem kills it. I noticed chrony allows to set a "precision" value, but from the documentation it was not clear if that's the jitter value and if it just enforces a minimum or sets it fixed. That option does not exist in ntpd. Michael ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Garmin LVC 18x jitter problem
Hello David, I trust the clock, because with constant temperature crystals perform pretty good. The clock is disciplined with PPS, but no adjustments happened during the measurement, so it runs with a constant frequency offset. It took a few days to reach that state. My point is that NMEA on this device does not jitter with a gauss distribution or similar, but in a way that causes the reasonable jitter calculation to give too low values. The jitter diagram shows why 0.1 as minimum distance suffices. The problem is, that this is only a workaround, as it effects the root dispersion. What I need is to bypass the NMEA jitter calculation only, or give a minimal value for that, to prevent it becoming a false ticker when the actual jitter is larger than the estimated jitter. In the diagrams that happens if the estimated jitter does not cover zero, because zero is the PPS/crystal clock with 4 us jitter (both estimated and actual). GPS reception was fine with good average SNR and a reasonable of used and visible satellites. TDOP was fine. I had this behavior before the upgrade from firmware 4.00 to 4.20, which is recommended due to the GPS week rollover. I did not have it when the GPS reception was worse and I had a lower average SNR and an overall higher NMEA jitter and worse TDOP. Michael ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
[ntp:questions] Garmin LVC 18x jitter problem
I use a Garmin LVC 18x with a Raspberry Pi, process NMEA with gpsd (SHM driver) and acess PPS by GPIO (kernel PPS driver). gpsd allows monitoring and passing PPS right into ntpd avoids gpsd conversion of PPS. Sometimes this works, but then there are times where both clocks are thrown out as falsetickers. I believe that's due to the strange jitter behaviour of the NMEA data. If I understood the clock selection right, then basically the measured jitter forms an interval around the offset and the intersection interval is the root dispersion. Should the NMEA clock have no intersection with the used PPS clock, it is a falseticker, but since PPS depends on it, PPS is so as well. I graphed offset and jitter from two days peerstats where I was locked on PPS (tos mindist 0.1) and the system ran stable without the need to adjust the crystal frequency at constant environment temperature. The first day was mostly fine, but in the middle of the second day, the GPS jittered really bad and there are many occasions where the jitter interval did not intersect with 0. The ntpd jitter estimation works as expected for a normal distribution, but the distribution is clearly different: http://www.moria.de/~michael/tmp/offset.svg http://www.moria.de/~michael/tmp/jitter.svg Using the tos mindist extends the intersection interval, but that effects the root dispersion. That's correct if it is needed to have an intersection between different clocks of low jitter, but in my case the problem is a wrong (too low) jitter estimation of a clock I only need as PPS reference. Is there any way to specify the precision manually, like fudge minjitter? Clearly the jitter suffices to keep the PPS clock running and I would like to have PPS determine the root dispersion, because the PPS clock has a jitter of 4 us. This problem seems to have come up a number of times in the past, but I never saw the root dispersion impact of tos mindist mentioned and I suspect in a number of cases configuring a minimal jitter would have been a better solution. Michael ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
