Re: [ntp:questions] ntpd - gpsd communication
On 2020-06-08, a...@avtodoria.ru wrote: > Currently i'm looking at ntp sources. as i said i have read data from > SHM at June 4th, now i'm trying to write them into SHM. What i see in > ntpq output: ... > You can see reftime is June 4th, but nevertheless offset is small. How > does it work? When i debug it small offset is before even clock select > algotirhm, so who calculates such small offset at June 8th It's not possible to replay old SHM data at a different time. Have a look at the SHM structure. There are receiveTimeStamp fields, which tell at what time the sample was captured (system time) and clockTimeStamp fields which tell what time the reference clock had at the time of the capture. The difference is the offset. If you want to replay an offset, you need to put in the SHM segment the current system time and the same time adjusted for the offset, so their difference is the offset as it was before. But this completely ignores the process that is controlling the clock, so I'm not sure what value this exercise has. -- Miroslav Lichvar ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] ntpd - gpsd communication
Yo Steven! On Mon, 8 Jun 2020 18:48:02 -0500 Steven Sommars wrote: > Sounds sort of plausible. However if the GPS doesn't have the > almanac, how can it get a fix? A GPS never uses the Almanac to compute a fix. A GPS uses the Ephemeris to compute a fix. Each satellite sends its own Ephemeris every 30 seconds. The Almanac is a reduced precision ephemeris only used to decide what sats should be in use. If you have a receiver that can receive many satellites at once then the Almanc is not needed at atll. But there are other things int subframe messages of use. > If this is a known issue, shouldn't > the GPS delay sending validity="A" or otherwise signal that the UTC > offset may be incorrect? Most do not. > This should be reproducible, correct? Yes, depending on the receiver model, very repeatable. > What kind of reset will cause > the GPS to lose the almanac information? Depends on the receiver type, and if/where is stores the current GPS-UTC offset. Some GPS have no power down storage, some have battery backed ram (BBR) and some have FLASH. > Many Rpi boards have > batteries or supercaps for power backup. Yup, but if the receiver is powered down long enough, the stored GPS-UTC offset will become out of data. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 Veritas liberabit vos. -- Quid est veritas? "If you can't measure it, you can't improve it." - Lord Kelvin pgpnrt6OlI_II.pgp Description: OpenPGP digital signature ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] ntpd - gpsd communication
Gary, Sounds sort of plausible. However if the GPS doesn't have the almanac, how can it get a fix?If this is a known issue, shouldn't the GPS delay sending validity="A" or otherwise signal that the UTC offset may be incorrect? This should be reproducible, correct? What kind of reset will cause the GPS to lose the almanac information? Many Rpi boards have batteries or supercaps for power backup. khronos.mikieboy.net is a Rpi+Uputronics board Steve On Mon, Jun 8, 2020 at 5:25 PM Gary E. Miller wrote: > Yo Steven! > > On Mon, 8 Jun 2020 17:06:55 -0500 > Steven Sommars wrote: > > > The off by N-second errors are often seen soon after NTP server > > initialization. > > This is common to all GPS based NTP servers. It is distinct from > long lasting off-by one errors that one version of gpsd could fall > victimm to. > > When a GPS starts up, it often uses the value for the current leap > second stored in the firmware. By "leap second" I mean the correction > from GPS tim to UTC time which is always in integer seconds. > > This stored leap second is often off by one or two seconds, or more, > for a while. > > After receiving the Almanac, which can take up to 25 minutes on some > older GPS, the current leap second offset is known. But now ntpd is > set in its ways, and will take a while to slew to the correct time. > > RGDS > GARY > --- > Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 > g...@rellim.com Tel:+1 541 382 8588 > > Veritas liberabit vos. -- Quid est veritas? > "If you can't measure it, you can't improve it." - Lord Kelvin > ___ > questions mailing list > questions@lists.ntp.org > http://lists.ntp.org/listinfo/questions > ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] ntpd - gpsd communication
Yo Steven! On Mon, 8 Jun 2020 17:06:55 -0500 Steven Sommars wrote: > The off by N-second errors are often seen soon after NTP server > initialization. This is common to all GPS based NTP servers. It is distinct from long lasting off-by one errors that one version of gpsd could fall victimm to. When a GPS starts up, it often uses the value for the current leap second stored in the firmware. By "leap second" I mean the correction from GPS tim to UTC time which is always in integer seconds. This stored leap second is often off by one or two seconds, or more, for a while. After receiving the Almanac, which can take up to 25 minutes on some older GPS, the current leap second offset is known. But now ntpd is set in its ways, and will take a while to slew to the correct time. RGDS GARY --- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 g...@rellim.com Tel:+1 541 382 8588 Veritas liberabit vos. -- Quid est veritas? "If you can't measure it, you can't improve it." - Lord Kelvin pgpIp6xIESBkg.pgp Description: OpenPGP digital signature ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] ntpd - gpsd communication
I monitor many NTP servers and have observed off-by-N second errors (N often 1, 2, 3) for years. Some of these errors are understood: fuzzing errors in ntpd, late NMEA arrival times. But many of these errors have unknown causes. These errors can recur on a given server with the same offset. On May 22 I noticed a system running ntpsec + GPSD ( khronos.mikieboy.net,) was in error by two seconds. The same system was also off by two seconds on January 15. I asked about this on the ntpsec mailing list, which happens to include several GPSD folks. Their answer was that a GPSD error caused the problem, but was fixed in GPSD 2.0 No bug description seems to exist, so I'm uncertain whether the errors I saw were related. The off by N-second errors are often seen soon after NTP server initialization. I lack detailed configuration information for many of the affected systems, unfortunately. On Mon, Jun 8, 2020 at 5:02 AM Miroslav Lichvar wrote: > On 2020-06-08, a...@avtodoria.ru wrote: > > понедельник, 8 июня 2020 г., 10:52:36 UTC+3 пользователь Miroslav > > Lichvar написал: > >> If you need something to report a large offset to ntpd via SHM, you > >> could try this program for testing leap seconds: > >> > >> https://github.com/mlichvar/leapshm > > > Thank you ver much. Please can you explain the params program uses. > > Second can be path to unix socket or something else starting not with > > "/" to use SHM. What about third param ? > > It's the number of seconds before the hardcoded leap second time (1 Jul > 2015 00:00:00 UTC) where the time fed by the program should start. E.g. > if it was 600, it would start 10 minutes before the midnight of the leap > second. You would probably want to start with a larger number (if you > don't want to test a leap second) and also change the LEAP_TIME constant > to a current date. > > -- > Miroslav Lichvar > > ___ > questions mailing list > questions@lists.ntp.org > http://lists.ntp.org/listinfo/questions > ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] ntpd - gpsd communication
On 2020-06-08, a...@avtodoria.ru wrote: > понедельник, 8 июня 2020 г., 10:52:36 UTC+3 пользователь Miroslav > Lichvar написал: >> If you need something to report a large offset to ntpd via SHM, you >> could try this program for testing leap seconds: >> >> https://github.com/mlichvar/leapshm > Thank you ver much. Please can you explain the params program uses. > Second can be path to unix socket or something else starting not with > "/" to use SHM. What about third param ? It's the number of seconds before the hardcoded leap second time (1 Jul 2015 00:00:00 UTC) where the time fed by the program should start. E.g. if it was 600, it would start 10 minutes before the midnight of the leap second. You would probably want to start with a larger number (if you don't want to test a leap second) and also change the LEAP_TIME constant to a current date. -- Miroslav Lichvar ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] ntpd - gpsd communication
On 2020-06-04, a...@avtodoria.ru wrote: > I want to add two another sources to fullfill the complect of three > generals)) but at first i want to understand how to cheat ntpd as gpsd > regularly does. I think i did all i need. But it looks like i did > something wrong in that test. Ntpd eats my data but doesn't touch > system time. And i can't understand why If you need something to report a large offset to ntpd via SHM, you could try this program for testing leap seconds: https://github.com/mlichvar/leapshm With some small modifications, it should be able to report any offset you want. -- Miroslav Lichvar ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] ntpd - gpsd communication
On 6/4/20 9:15 PM, a...@avtodoria.ru wrote: > Hello!! > > I have time jumps made by ntpd because gpsd sometimes puts wrong data to SHM > because of wrong data from gps receiver(very bad chips) > > I have only one time source in ntp.conf — 127.127.28.1 with PPS enabled > > I want to add another two internet sources for stability but at first i want > to emulate wrong data from gpsd to see how ntpd will make this time jump. > > As first step i read data from SHM when gpsd worked correctly at 14 p.m. > today UTC+3 for 30 minutes > > And after that i stopped gpsd and launched my own binary at 17:30 p.m. UTC+3 > writing that old data to SHM. I expected the great offset and the time jump > after some time(as it was when receiver lied) but what i saw was: > - without binary launched ntpd had no updates — it’s correct (no data — no > action) > - with binary launched ntpd had a little offset for all 30 minutes without > any attempt to correct system time > > please give me some help!! Maybe i don’t understand ntpd — gpsd communication > correctly or smth else > > Thank you in advance > There's another way to communicate between gpsd and ntpd, using driver 46 (gpsd-json). It probably won't help with a really bad receiver, but it is a bit more sophisticated than the SHM memory slot, even if I say so myself. Stale data is much less of a problem with this driver. It might give different/better results, but I won't make promises here. Cheers, Pearly ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] ntpd - gpsd communication
Am 05.06.2020 um 09:53 schrieb ah@***.ru: пятница, 5 июня 2020 г., 10:25:24 UTC+3 пользователь Uwe Klein написал: Am 05.06.2020 um 08:36 schrieb ah@***.ru: This sounds more like bit errors in serial reception. ( i.e errors coming up in all magnitudes.) That is reception via NMEA? do you check the NMEA checksum ( Portion after * afair)? Thank you for response. Yes, that is reception via NMEA. No i didn't check checksum, i think gpsd should check it, am i wrong? I just write to log data from SHM when it's all OK. And try to use this correct log after some time to make ntpd change the time. If you think i do something not correct please tell me, i'll try to do something else can you enable data logging and produce logs from the GPS instances that produce errors? ( all sentences received over some time.) Just from a single sytem. As an alternate take a terminal application configured for 4800baud and for the right serial port and log data via that path. never used gpsd put did my own solution years ago for a special use case: https://wiki.tcl-lang.org/page/NTP+plugin+for+a+detachable+nmea+GPS+source ( one app sources NMEA from the device and reflects that into UDP packets.) the other part is sinking UDP packets and funneling those into NTPD. kind of a primitive gpsd replacement ) Uwe I launched gpspipe -R jn all systems, but it's hard to catch it again. I can be not reproducing for a long time Thank you for your decision - did you write it into SHM ? I will see it asaic SHM :: no. it creates an "active" pty device for ntpd to attach to. For ntpd this looks like a regular serial connection. Uwe ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] ntpd - gpsd communication
Am 05.06.2020 um 00:10 schrieb a...@avtodoria.ru: The typical behaviour in case of time jump is that suddenly we have time in gpspipe jumped forth or back for [some seconds - some years]. This sounds more like bit errors in serial reception. ( i.e errors coming up in all magnitudes.) That is reception via NMEA? do you check the NMEA checksum ( Portion after * afair)? Uwe ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] ntpd - gpsd communication
пятница, 5 июня 2020 г., 10:25:24 UTC+3 пользователь Uwe Klein написал: > Am 05.06.2020 um 08:36 schrieb ah@***.ru: > >> This sounds more like bit errors in serial reception. > >> ( i.e errors coming up in all magnitudes.) > > > >> That is reception via NMEA? > >> do you check the NMEA checksum ( Portion after * afair)? > > > > Thank you for response. Yes, that is reception via NMEA. No i didn't check > > checksum, i think gpsd should check it, am i wrong? > > > > I just write to log data from SHM when it's all OK. And try to use this > > correct log after some time to make ntpd change the time. If you think i do > > something not correct please tell me, i'll try to do something else > > > can you enable data logging and produce logs from the GPS instances that > produce errors? ( all sentences received over some time.) > > Just from a single sytem. > > As an alternate take a terminal application configured for 4800baud > and for the right serial port and log data via that path. > > never used gpsd put did my own solution years ago for a special use case: > > https://wiki.tcl-lang.org/page/NTP+plugin+for+a+detachable+nmea+GPS+source > > ( one app sources NMEA from the device and reflects that into UDP packets.) > the other part is sinking UDP packets and funneling those > into NTPD. kind of a primitive gpsd replacement ) > > Uwe I launched gpspipe -R jn all systems, but it's hard to catch it again. I can be not reproducing for a long time Thank you for your decision - did you write it into SHM ? I will see it asaic ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] ntpd - gpsd communication
> What is the wrong data? Ie, how much of an offset does the wrong data > imply? > What does wrong data mean? Do you mean that the PPS is wrong? that of > course has no time attached so can be wrong by only up to 1 second. > Or do you mean that the time reported by the NMEA sentences is wrong? > How wrong? 1 sec? 100 days? We have a lot of computers with different hardware(we can mean it as different generations of equipment). Among this equipment there are different receivers, the last portion of them are stable but the first is not. I have already requested logs with gpspipe -R from system administrators, but we lost those with wrong behaviour and now we are waiting the new event. Try to believe me please... The typical behaviour in case of time jump is that suddenly we have time in gpspipe jumped forth or back for [some seconds - some years]. Ntpd can't believe to that jump but because this source is the only after some time (about 5-10 minutes) ntpd hardly set system time to time in SHM. About 10-15 minutes system time is wrong but ntpd characterisitics are good(offset is less than 1 ms, little jitter etc). We have situation named 'ntpd has been cheated'. After that period of time there is reverse jump in gpspipe -R log. And ntpd again slowly starts to believe in another jump. After all situation becomes OK. > Note that if it really is your GPS that is misbehaving, why not just buy > another board. They are cheap. The Adafruit gps is only about $50. > Your tearing out your hair is probably worth more than that. I agree with you it is the best solution. But managers solved to try to fix it by programmers hands at first. So here am i )) I want to add two another sources to fullfill the complect of three generals)) but at first i want to understand how to cheat ntpd as gpsd regularly does. I think i did all i need. But it looks like i did something wrong in that test. Ntpd eats my data but doesn't touch system time. And i can't understand why ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] ntpd - gpsd communication
пятница, 5 июня 2020 г., 10:04:44 UTC+3 пользователь William Unruh написал: > On 2020-06-05, a...@avtodoria.ru wrote: > >> This sounds more like bit errors in serial reception. > >> ( i.e errors coming up in all magnitudes.) > > > >> That is reception via NMEA? > >> do you check the NMEA checksum ( Portion after * afair)? > > > > Thank you for response. Yes, that is reception via NMEA. No i didn't check > > checksum, i think gpsd should check it, am i wrong? > > > > I just write to log data from SHM when it's all OK. And try to use this > > correct log after some time to make ntpd change the time. If you think i do > > something not correct please tell me, i'll try to do something else > > > > You are tracking the wrong problem. The problem is NOT ntpd. The problem > is the time you are getting from gps/gpsd/shm. That is where you should > be looking, not ntpd. ntd has been used for about 40 years now. It > works. Similarly gpsd. Your boss is telling you to fix something that is > not broken, wasting whatever salary they are paying you. Do they really > have solittle for you to do that is useful? > You have not told us what operating system you are using. Or anything > about your systems, except some are old (3mo?, 3yr? 10 yr? 30 yr?) No-no-no... i didn't tell ntpd or gpsd are wrong.. Sorry, if i said something like that, i have no enough practice in english communication to be confident i make right language constructions :) I agree with you receivers are bad, i agree they receive wrong data or there are some problem in serial port or smth else But. What i want to say is that this wrong situations make ntpd to do some changes to system time. And i want to simulate that situation. That's all. My system is ALT Linux 6.0.0 Centaurus (Cheiron), 4.4.135-std-pae-alt1 ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] ntpd - gpsd communication
> This sounds more like bit errors in serial reception. > ( i.e errors coming up in all magnitudes.) > That is reception via NMEA? > do you check the NMEA checksum ( Portion after * afair)? Thank you for response. Yes, that is reception via NMEA. No i didn't check checksum, i think gpsd should check it, am i wrong? I just write to log data from SHM when it's all OK. And try to use this correct log after some time to make ntpd change the time. If you think i do something not correct please tell me, i'll try to do something else ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] ntpd - gpsd communication
On 2020-06-05, a...@avtodoria.ru wrote: >> This sounds more like bit errors in serial reception. >> ( i.e errors coming up in all magnitudes.) > >> That is reception via NMEA? >> do you check the NMEA checksum ( Portion after * afair)? > > Thank you for response. Yes, that is reception via NMEA. No i didn't check > checksum, i think gpsd should check it, am i wrong? > > I just write to log data from SHM when it's all OK. And try to use this > correct log after some time to make ntpd change the time. If you think i do > something not correct please tell me, i'll try to do something else > You are tracking the wrong problem. The problem is NOT ntpd. The problem is the time you are getting from gps/gpsd/shm. That is where you should be looking, not ntpd. ntd has been used for about 40 years now. It works. Similarly gpsd. Your boss is telling you to fix something that is not broken, wasting whatever salary they are paying you. Do they really have solittle for you to do that is useful? You have not told us what operating system you are using. Or anything about your systems, except some are old (3mo?, 3yr? 10 yr? 30 yr?) ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] ntpd - gpsd communication
On 2020-06-04, a...@avtodoria.ru wrote: > Hello!! > > I have time jumps made by ntpd because gpsd sometimes puts wrong data to SHM > because of wrong data from gps receiver(very bad chips) What is the wrong data? Ie, how much of an offset does the wrong data imply? What does wrong data mean? Do you mean that the PPS is wrong? that of course has no time attached so can be wrong by only up to 1 second. Or do you mean that the time reported by the NMEA sentences is wrong? How wrong? 1 sec? 100 days? > > I have only one time source in ntp.conf — 127.127.28.1 with PPS enabled > > I want to add another two internet sources for stability but at first i want > to emulate wrong data from gpsd to see how ntpd will make this time jump. > > As first step i read data from SHM when gpsd worked correctly at 14 p.m. > today UTC+3 for 30 minutes > > And after that i stopped gpsd and launched my own binary at 17:30 p.m. UTC+3 > writing that old data to SHM. I expected the great offset and the time jump > after some time(as it was when receiver lied) but what i saw was: > - without binary launched ntpd had no updates — it’s correct (no data — no > action) > - with binary launched ntpd had a little offset for all 30 minutes without > any attempt to correct system time > > please give me some help!! Maybe i don’t understand ntpd — gpsd communication > correctly or smth else Note that if it really is your GPS that is misbehaving, why not just buy another board. They are cheap. The Adafruit gps is only about $50. Your tearing out your hair is probably worth more than that. > > Thank you in advance ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
