Re: [ntp:questions] NTP & PPS, part 2 ;)
Phil W Lee wrote: David Taylor considered Tue, 16 Dec 2014 14:26:49 + the perfect time to write: It would be helpful if the output from ntpq -crv showed the OS on which NTP was running, as well as the OS on which it was built. I've mentioned this before but didn't push it strongly. There is a query which returns the information you want, as it's used by the Meinerg NTP monitor for windows (right hand button between the status window and "Refresh interval". This is just a subset of the information you get from "ntpq -c rv", e.g.: associd=0 status=0615 leap_none, sync_ntp, 1 event, clock_sync, version="ntpd 4.2.6p5@1.2349-o Jul 30 11:55:08 (UTC+02:00) 2012 (2)", processor="x86", system="Windows", leap=00, stratum=2, precision=-10, rootdelay=0.015, rootdisp=4.447, refid=172.16.3.9, reftime=d83bc128.df638c2d Wed, Dec 17 2014 9:45:28.872, clock=d83bc157.26a94acd Wed, Dec 17 2014 9:46:15.151, peer=29231, tc=6, mintc=3, offset=0.171, frequency=500.000, sys_jitter=0.000, clk_jitter=0.985, clk_wander=0.003 telling on which system ntpd is running, *not* on which system it has been built. Martin ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Number of Stratum 1 & Stratum 2 Peers
Phil W Lee wrote: Martin Burnicki considered Tue, 16 Dec 2014 12:56:15 +0100 the perfect time to write: William Unruh wrote: The importance of trades is usually a before/after. And UTC TAI, GPS all have exactly the same definition of before and after. Of course if one time was in UTC and the otehr in TAI, that could well be successfully argued. From a technical point I absolutely agree with you. However, there have been discussions (IIRC on the TZ or leapsecs mailing lists) where folks tried to explain that even though the difference between TAI and UTC is just an integral number of seconds, TAI could not even be used as a replacement for UTC without leap seconds, just due to specific wordings in certain documents. Leap seconds seem to be a real mess in the IT world. It would be useful if the way of inserting a leap-second was set in a standard, in such a way that time continued at a set rate (maybe by slewing at a set percentage or PPM). If that could be achieved, it would remove many of the objections to leap-seconds. It might be difficult to thrash out in practice though. I know that officially at present there is an additional second between 23:59:60 and 00:00:00, but no time recording system that I know of has the ability to record times between 23:59:60 and 00:00:00 correctly (despite such times existing since 1st Jan 1961, which must surely pre-date any software currently in use), which is a necessary requirement if the second is to be inserted exactly as currently specified and time continued forward (so that events are recorded in the correct order). Does anyone know of any software which will record times during that additional second accurately, e.g. as 23:59:60.789? Yes, different software from Meinberg. ;-) The main problem is that the underlying system time (often POSIX, which just counts seconds since an epoch) has the *same* time stamp art the beginning and end of the leap second. In order to do the conversion correctly you need to know if the current second is the leap second, or not, i.e. you need some status flag in addition to the raw (e.g. POSIX) timestamp. This is basically similar to what you have at the end of DST, where (in local time) a whole hour is passed twice. You need to know the DST status to determine unambiguously if it's the first or the second turn. Is there any realistic prospect of forcing software to comply with a time standard which includes times between 23:59:60 and 00:00:00? Now, you can either stipulate that all software including operation systems recognise times during that additional second - which would require re-writing the time functions of most of the worlds software to recognise and record times >23:59:60 < 00:00:00 (but only if a leap second has been legally notified), or you can agree to insert the additional second more gradually by clock slewing (but at what rate would have to be agreed). Clock slewing would require much more change to international agreements, but would require far less work on re-writing software, and would actually relate better to the real world, which is annoyingly both analogue AND irregular:-) Or the second itself could be redefined to take account of the actual speed the earth rotates - but that might be problematic for the scientists, as we'd likely have to keep doing it as the earth slows. I certainly think we need to deal with the problem better than we do at the moment. A main reason for the problems with leap seconds is that POSIX has just ignored them when they defined their standards on timekeeping. There has been a proposal from David Mills many years ago where the time during an inserted leap second increases only *very* slowly during the leap second, so monotonicity of time stamps is kept. However, most algorithms, or API calls returning a time stamp plus consistent status information require longer execution time than just returning a time stamp, e.g. just a 64 bit number. So the easy way is often preferred over the accurate way. I've collected some information on leap seconds in a paper which you can find here: http://www.meinbergglobal.com/english/info/#whitepaper Especially: "Technical Aspects of Leap Second Propagation and Evaluation" http://www.meinbergglobal.com/download/burnicki/Technical%20Aspects%20of%20Leap%20Second%20Propagation%20and%20Evaluation.pdf This may not be complete but covers many aspects of leap second handling. Martin ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Number of Stratum 1 & Stratum 2 Peers
Martin Burnicki wrote: > The main problem is that the underlying system time (often POSIX, which > just counts seconds since an epoch) has the *same* time stamp art the > beginning and end of the leap second. > > In order to do the conversion correctly you need to know if the current > second is the leap second, or not, i.e. you need some status flag in > addition to the raw (e.g. POSIX) timestamp. > > This is basically similar to what you have at the end of DST, where (in > local time) a whole hour is passed twice. You need to know the DST > status to determine unambiguously if it's the first or the second turn. Yes, but in situations where you care about that you store just the system time, and the conversion to local time can be repeated later. Thanks to the powerful conversion routines in glibc this can be done even way in the past (because it remembers DST rules from previous years). Unfortunately, the same mechanism isn't used for leap seconds. There would be no problem at all when the system time ticked in TAI and the addition of the leap seconds is done via some rule table similar to the local time rules. ntpd would not even be involved in the problem. > A main reason for the problems with leap seconds is that POSIX has just > ignored them when they defined their standards on timekeeping. That is the problem. And probably also the decision in Unix to count UTC seconds since Jan 1st, 1970 (instead of TAI seconds). Back then it did not matter that much in a timesharing system. The system clock was mainly a convenience feature, not critical to anything. (it usually had to be set on every boot and this was done by typing the time seen on the operator's wristwatch, hence the term "wristwatch time") ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Number of Stratum 1 & Stratum 2 Peers
Phil W Lee wrote: Martin Burnicki considered Tue, 16 Dec 2014 14:23:15 +0100 the perfect time to write: Harlan Stenn wrote: An alternative is that we get enough support to advance NTF's General Timestamp API, and then we can run systems on either TAI or UTC and these conversions will happen automatically. Since timescale files in the GTSAPI are "versioned", one could still use an obsolete leapsecond file, and while those UTC timestamps would be "wrong" if a new leapsecond was added, these timestamps would be correctable when a new version of the UTC timescale file was available. Hm, that may not really help if the API returns a wrong UTC time stamp which is then used to set the system time wrong. The tzdist protocol could also be helpful here to provide the information required to do the conversion correctly. An expiration date could be used for versioning. Martin You don't need an expiry date if you have a version number and/or an authoritative source for any new version that may be available - you just compare the two, and use the newest available. Yes you do. With only a version number "consumers" like ntpd would not be able to know if the information is outdated, or not. Of course, if leap seconds should be abolished it would be useful to support a pseudo expiration date meaning "until further notice". As long as the IERS stays at the same URL, you could just use their file at http://hpiers.obspm.fr/eoppc/bul/bulc/Leap_Second.dat (although it would be useful if that file was more complete, with a version number and checksum). This is once more a different file format than the format used by tzdata, or NIST/NTP. :-( A service like a tzdist client, or a simple script which might look for and download updated files, could report an error if the URL is not reachable, and thus it can't even *check* if a new "version" of the file is available. However, similarly as not every tiny NTP client node should query the time directly from NIST and similar servers but should use pool servers instead, not every tiny embedded system should try to download a leap second file directly from the primary server. If they use secondary servers an older version of the file may me available, but outdated. No way to check this without an expiration date. There are companies with a whole (sub-)network without access to the internet, so it may be required to update DST rules and leap second information manually. An easy way to do this could be to set up a tzdist or FTP (or whatever) server which can provide the internal clients with the update. If no one cares about those updates then applications like ntpd can output a warning if the expiration date has been passed. With only a version information this isn't possible. It would also be useful if they used SSL, and changed the url to https://etc. Agreed. Martin ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Number of Stratum 1 & Stratum 2 Peers
Harlan Stenn wrote: Martin Burnicki writes: Harlan Stenn wrote: An alternative is that we get enough support to advance NTF's General Timestamp API, and then we can run systems on either TAI or UTC and these conversions will happen automatically. Since timescale files in the GTSAPI are "versioned", one could still use an obsolete leapsecond file, and while those UTC timestamps would be "wrong" if a new leapsecond was added, these timestamps would be correctable when a new version of the UTC timescale file was available. Hm, that may not really help if the API returns a wrong UTC time stamp which is then used to set the system time wrong. Anything on the box that uses UTC in this situation will be using an outdated UTC timescale, and when any newer versions are made available the library API will handle the conversion. Think of the case where you want to set an event for a future time - if you want to tag something with an absolute time of 10am UTC a year in the future, there's no way to know if there will be a leap second in there or not. So as we get closer to the actual date, if the UTC timescale gets updated the absolute UTC timestamp will be properly adjusted as soon as we know. I don't know if you are subscribed to the tzdist mailing list, on which there is actually a similar discussion: Imagine you set up an event for April 2015 today, but you just don't know if DST will be in effect at that time, or not, just because the politicians haven't made the decision today. How will you handle this? It may not be helpful if you get updated DST rules the day *after* the scheduled event. On the other hand, when the event time approaches and you have the chance to get a DST rule update before the event time everything may be fine, if a decision is taken either to keep the UTC time *or* the local time for this event. Since UTC is actually used in most computers to keep the system time and schedule events based on an accurate UTC time it may not be helpful to do this with post-processing. Martin ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Number of Stratum 1 & Stratum 2 Peers
Phil W Lee wrote: Martin Burnicki considered Tue, 16 Dec 2014 12:48:57 +0100 the perfect time to write: However, there is an important limitation: the tzdata version of the leap second file is missing an expiration date, so even if a program like ntpd could use this file directly it would never know if no more leap second has been scheduled after the last one mentioned in this file, or if the file just hasn't been updated recently enough. If it compared the file it has with the one available from tzdata, it could see if there was a difference - it's not exactly huge. Even better would be if a checksum or version number was available as well, so that could just be compared to the current one, and a new one only downloaded if required. This presumes you have access to version which is known to be current, but fails if it is not the case. See also my other reply. Martin ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Number of Stratum 1 & Stratum 2 Peers
Rob wrote: Martin Burnicki wrote: The main problem is that the underlying system time (often POSIX, which just counts seconds since an epoch) has the *same* time stamp art the beginning and end of the leap second. In order to do the conversion correctly you need to know if the current second is the leap second, or not, i.e. you need some status flag in addition to the raw (e.g. POSIX) timestamp. This is basically similar to what you have at the end of DST, where (in local time) a whole hour is passed twice. You need to know the DST status to determine unambiguously if it's the first or the second turn. Yes, but in situations where you care about that you store just the system time, and the conversion to local time can be repeated later. Thanks to the powerful conversion routines in glibc this can be done even way in the past (because it remembers DST rules from previous years). Agreed, for this case it works, due to effort of the tzdata maintainers. Unfortunately, the same mechanism isn't used for leap seconds. There would be no problem at all when the system time ticked in TAI and the addition of the leap seconds is done via some rule table similar to the local time rules. ntpd would not even be involved in the problem. Also agreed. However, actually ntpd expects UTC, not TAI. In spite of this it may eventually work if you set up an NTP server where the system time runs on TAI and you use one of the "right" time zones which are also part of the tzdata package. I haven't tried this. You'd have to take care that ntpd does *not* announce leap seconds in this case, but you'd have to update the leap second information anyway, since otherwise the conversion done according to the "right" time zone rules would be wrong. ;-) A main reason for the problems with leap seconds is that POSIX has just ignored them when they defined their standards on timekeeping. That is the problem. And probably also the decision in Unix to count UTC seconds since Jan 1st, 1970 (instead of TAI seconds). Back then it did not matter that much in a timesharing system. The system clock was mainly a convenience feature, not critical to anything. (it usually had to be set on every boot and this was done by typing the time seen on the operator's wristwatch, hence the term "wristwatch time") Again agreed. Using TAI for sytem time and a current leap second table would allow to convert TAI to UTC unambiguously. I think that's what the conversion routines do for the "right" timezones. On the other hand, maybe you have seen the discussions elsewhere why you can't use TAI for the system time for hypothetical reasons. Or at least you can't call it "TAI". Martin ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Number of Stratum 1 & Stratum 2 Peers
Martin Burnicki wrote: > Imagine you set up an event for April 2015 today, but you just don't > know if DST will be in effect at that time, or not, just because the > politicians haven't made the decision today. How will you handle this? > > It may not be helpful if you get updated DST rules the day *after* the > scheduled event. > > On the other hand, when the event time approaches and you have the > chance to get a DST rule update before the event time everything may be > fine, if a decision is taken either to keep the UTC time *or* the local > time for this event. > > Since UTC is actually used in most computers to keep the system time and > schedule events based on an accurate UTC time it may not be helpful to > do this with post-processing. It depends on what kind of event it is. When it is a "human event" like a meeting, it is customary to plan these events in local time. So they should be stored in local time with a specified timezone. What UTC time this is will depend on the local time rules at the actual event time. It will still be tricky when you plan a meeting at 02:30 on the sunday of the DST change from summer->winter time. You cannot add a DST flag with the event because you cannot know if there will be a DST change at that time. Of course that is why the DST change is done at a moment where such problems are unlikely, and not for example at 12:00 on the first day of the month. Some calendering software gets this wrong, yes. E.g. by converting the specified local time to UTC time and storing that. This has caused problems in the past for well-known calendering software, I don't know if that has been fixed. For "technical" events it may be different, and storing them at the UTC time could be better. But scheduling software (Windows Scheduler, Unix cron) usually uses local time as well. ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Number of Stratum 1 & Stratum 2 Peers
Martin Burnicki wrote: >> Unfortunately, the same mechanism isn't used for leap seconds. >> There would be no problem at all when the system time ticked in TAI >> and the addition of the leap seconds is done via some rule table similar >> to the local time rules. ntpd would not even be involved in the problem. > > Also agreed. However, actually ntpd expects UTC, not TAI. Of course it is a hypothetical situation. When the Unix developers would have had the foresight to define their clock in TAI rather than UTC, the NTP designer probably would have used TAI as well, and instead of the leap second handling there maybe would have been some field to broadcast the current UTC-TAI offset. GPS actually does it that way. ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] NTP & PPS, part 2 ;)
David Taylor writes: > On 16/12/2014 13:48, Martin Burnicki wrote: > > William Unruh wrote: > [] > >> And since at build time, one has things called "configure" which CAN run > >> tests on the build system, one could easily enable or disable it then. > >> But since as we all know ntpd tends to built on one system and used on > >> myriads of other systems, the code should run tests when running, not > >> when compiling, to see if the api actually works. That is the way all > >> other "optional" stuff in all other packages works. After all, if there > >> is no nmea gps attached to the computer, trying to use the nmea driver > >> will not work. > > > > That's exactly what I meant. > > > > Martin > > It would be helpful if the output from ntpq -crv showed the OS on which > NTP was running, as well as the OS on which it was built. I've > mentioned this before but didn't push it strongly. autoconf gives us "build" and "host" variables. The "build" system describes the type of system where the package is configured and compiled. The "host" system describes the type of system on which the package runs. Of course, this value can be overridden, and often is. STR_SYSTEM is already the "host" value, and it is reported thru the 'system' variable. Some systems report the cpu-vendor-os triple there, some don't. See also http://bugs.ntp.org/2260 . H ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Number of Stratum 1 & Stratum 2 Peers
Rob wrote: Martin Burnicki wrote: Unfortunately, the same mechanism isn't used for leap seconds. There would be no problem at all when the system time ticked in TAI and the addition of the leap seconds is done via some rule table similar to the local time rules. ntpd would not even be involved in the problem. Also agreed. However, actually ntpd expects UTC, not TAI. Of course it is a hypothetical situation. When the Unix developers would have had the foresight to define their clock in TAI rather than UTC, the NTP designer probably would have used TAI as well, and instead of the leap second handling there maybe would have been some field to broadcast the current UTC-TAI offset. Agreed. GPS actually does it that way. Yes, I know. I've by myself written a set of routines which decodes and evaluates the data broadcasted by the GPS satellites. IMO the GPS system designers have made quite a number of wise decisions, e.g. letting the GPS time simply increase monotonically, which is, from a technical/usage point of view, similar to TAI. AFAIK the GLONASS satellites use UTC instead, which causes an interruption in the sequence of data frames whenever a leap second occurs. Martin ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Number of Stratum 1 & Stratum 2 Peers
Phil W Lee writes: > Leap seconds seem to be a real mess in the IT world. > It would be useful if the way of inserting a leap-second was set in a > standard, in such a way that time continued at a set rate (maybe by > slewing at a set percentage or PPM). If that could be achieved, it > would remove many of the objections to leap-seconds. > It might be difficult to thrash out in practice though. NTF's General Timestamp API is ready to address this. We just need to get enough support to get the ball rolling. Some timescales have leapseconds, some do not. If a timescale has leap seconds, there are different ways it can be applied. As long as these choices are "defined" it's possible to usefully 'map' a timestamp taken in one (version of one) timescale to a timestamp in a different (version of a) timescale. > I know that officially at present there is an additional second > between 23:59:60 and 00:00:00, but no time recording system that I > know of has the ability to record times between 23:59:60 and 00:00:00 > correctly (despite such times existing since 1st Jan 1961, which must > surely pre-date any software currently in use), which is a necessary > requirement if the second is to be inserted exactly as currently > specified and time continued forward (so that events are recorded in > the correct order). > Does anyone know of any software which will record times during that > additional second accurately, e.g. as 23:59:60.789? There are implementations of this that can make it impossible. Nothing much can be done about things that occur during that extra second. If the implementation does something more ... accommodating, it is possible to track and map these events. Some systems will "slow time" by half, so the leap second occurs over 2 seconds. Some will slew it over 2 seconds. Google's "leap smear" does it unevenly over a 24-hour period. > Is there any realistic prospect of forcing software to comply with a > time standard which includes times between 23:59:60 and 00:00:00? I think so. And folks can always choose to track time using TAI and then map those events to the appropriate UTC. > Now, you can either stipulate that all software including operation > systems recognise times during that additional second - which would > require re-writing the time functions of most of the worlds software > to recognise and record times >23:59:60 < 00:00:00 (but only if a leap > second has been legally notified), or you can agree to insert the > additional second more gradually by clock slewing (but at what rate > would have to be agreed). > Clock slewing would require much more change to international > agreements, but would require far less work on re-writing software, > and would actually relate better to the real world, which is > annoyingly both analogue AND irregular:-) > Or the second itself could be redefined to take account of the actual > speed the earth rotates - but that might be problematic for the > scientists, as we'd likely have to keep doing it as the earth slows. > > I certainly think we need to deal with the problem better than we do > at the moment. Please support/participate in NTF's GTSAPI Consortium. -- Harlan Stenn http://networktimefoundation.org - be a member! ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] NTP & PPS, part 2 ;)
Paul writes: > On Tue, Dec 16, 2014 at 10:10 AM, Martin Burnicki < > martin.burni...@meinberg.de> wrote: > > > OK, but what is the problem in using these IOCTLs directly from within > > ntpd, via wrapper functions or directly? Several refclock drivers do so. > > You'll have to ask Harlan. Useful discussion and patches would be nice. H ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Number of Stratum 1 & Stratum 2 Peers
Phil W Lee writes: > Martin Burnicki considered Tue, 16 Dec > 2014 14:23:15 +0100 the perfect time to write: > > >Harlan Stenn wrote: > >> An alternative is that we get enough support to advance NTF's General > >> Timestamp API, and then we can run systems on either TAI or UTC and > >> these conversions will happen automatically. > >> > >> Since timescale files in the GTSAPI are "versioned", one could still use > >> an obsolete leapsecond file, and while those UTC timestamps would be > >> "wrong" if a new leapsecond was added, these timestamps would be > >> correctable when a new version of the UTC timescale file was available. > > > >Hm, that may not really help if the API returns a wrong UTC time stamp > >which is then used to set the system time wrong. While a "traditional" timestamp would have problems, if the system time uses the GTSAPI structure the "wrong" UTC timestamp would not be "wrong", it would just be using the older timescale version. These can be remapped as needed. This is equivalent to the case where we want to "do something in the future" with an absolute UTC timestamp. > >The tzdist protocol could also be helpful here to provide the > >information required to do the conversion correctly. An expiration date > >could be used for versioning. > > > >Martin > > You don't need an expiry date if you have a version number and/or an > authoritative source for any new version that may be available - you > just compare the two, and use the newest available. As long as you have the timescale versions for the timestamps you are tracking, one can do the conversions. > As long as the IERS stays at the same URL, you could just use their > file at http://hpiers.obspm.fr/eoppc/bul/bulc/Leap_Second.dat > (although it would be useful if that file was more complete, with a > version number and checksum). > It would also be useful if they used SSL, and changed the url to > https://etc. There are several timescales available there. Right now it's looking like we can use TAI as a canonical timescale and map that into different UTC timescales (they "version" every 6 months' or so) and we can do similarly with IERS-A timescales (which "version" every week). Then there are folks who will apply leap-seconds differently. This is to be expected, and must be dealt with. From what we've seen, this is doable. -- Harlan Stenn http://networktimefoundation.org - be a member! ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] ATOM driver not working on Linux
Harlan Stenn wrote: Martin Burnicki writes: Harlan Stenn wrote: David Taylor writes: I have a newly installed 64-bit Linux Debian 7.7 system where I am trying to bring up NTP with PPS support. Using ppstest /dev/pps0 I get the expected assert messages. I have gpsd configured and working. Having configured and compiled NTP I get the error message: clock type 22 invalid That's the parse refclocks. Nope. The parse clocks are type 8. I caught that mistake already and posted about it. Yep. Sorry, I should have read the whole thread first. Did you configure with --enable-parse-clocks ? That should not be required to have support for the ATOM driver be compiled in. Agreed. On the other hand, I've just been in contact with someone else claiming that support for driver 28 was not been compiled in unless --enable-parse-clocks was passed to ./configure. I couldn't duplicate this, yet, but we should have a closer look at this before RC becomes the next release. I'm not sure how to do that yet, so if anybody has an idea about it please speak up or do some testing now. In the mean time I've run a few quick tests on different current Linux systems and everything worked as expected, i.e. if timepps.h is detected then PPS support is compiled in, but otherwise not. Martin ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Default total number of servers NTP wants to have when using pool .....
I'd love to see discussion about "what should the default number of servers queried be for the 'pool' directive?" There is clearly a tradeoff, and I'm inclined to say that "between 5 and 9" is probably a good number. It's certainly easy enough for folks to join the pool, and I'd like to make that easier. Having said that, this is clearly a case where the "current reality" can change, and this is something the web-based ntp.conf generator (that NTF wants to make available) should easily be able to handle. We need some support to get this going, volunteers and/or $ will do it. -- Harlan Stenn http://networktimefoundation.org - be a member! ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] NTP & PPS, part 2 ;)
Martin Burnicki writes: > This is just a subset of the information you get from "ntpq -c rv", e.g.: > > associd=0 status=0615 leap_none, sync_ntp, 1 event, clock_sync, > version="ntpd 4.2.6p5@1.2349-o Jul 30 11:55:08 (UTC+02:00) 2012 (2)", > processor="x86", system="Windows", leap=00, stratum=2, precision=-10, > rootdelay=0.015, rootdisp=4.447, refid=172.16.3.9, > reftime=d83bc128.df638c2d Wed, Dec 17 2014 9:45:28.872, > clock=d83bc157.26a94acd Wed, Dec 17 2014 9:46:15.151, peer=29231, > tc=6, mintc=3, offset=0.171, frequency=500.000, sys_jitter=0.000, > clk_jitter=0.985, clk_wander=0.003 > > telling on which system ntpd is running, *not* on which system it has > been built. When I have more time I'll look at how 'system' is being set - that *should* be the '$host' CPU-VENDOR-OS value. If that variable is too inconsistent, I'm game to add the 'host-cvo' and 'build-cvo' values (or host_cvo and build_cvo, if those names would be better). If cvo_host and cvo_build would be better, that's fine too. H ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Number of Stratum 1 & Stratum 2 Peers
Martin Burnicki wrote: > IMO the GPS system designers have made quite a number of wise decisions, > e.g. letting the GPS time simply increase monotonically, which is, from > a technical/usage point of view, similar to TAI. That decision was wise. The decision to express date in weeks and put it in a 10-bit field was not so wise. Oh well. > AFAIK the GLONASS satellites use UTC instead, which causes an > interruption in the sequence of data frames whenever a leap second occurs. I think GLONASS uses "Moscow local time". I read that Galileo also has its own time scale (GST) synchronized to TAI and it broadcasts the time offset to GPS time expressed in nanoseconds. ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Number of Stratum 1 & Stratum 2 Peers
Martin Burnicki writes: > Rob wrote: > > Unfortunately, the same mechanism isn't used for leap seconds. > > There would be no problem at all when the system time ticked in TAI > > and the addition of the leap seconds is done via some rule table similar > > to the local time rules. ntpd would not even be involved in the problem. > > Also agreed. However, actually ntpd expects UTC, not TAI. One of the things I'm thinking about for NTP5 is systems that choose to run in TAI and not UTC. This is a natural tie-in with NTF's General Timestamp API, as we need to know what timescale the remote system is using. This woud also handle the case of maintaining sync with a system that chooses to implement Google's leap smear, or even one that wanted to run on IERS-A time. -- Harlan Stenn http://networktimefoundation.org - be a member! ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] NTP & PPS, part 2 ;)
Harlan Stenn wrote: > Paul writes: >> On Tue, Dec 16, 2014 at 10:10 AM, Martin Burnicki < >> martin.burni...@meinberg.de> wrote: >> >> > OK, but what is the problem in using these IOCTLs directly from within >> > ntpd, via wrapper functions or directly? Several refclock drivers do so. >> >> You'll have to ask Harlan. > > Useful discussion and patches would be nice. The code does not have to be patched because it already works, you only need to move the timepps.h file into the distributed source. It provides the wrapper functions between the existing refclocks and the IOCTLs. ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] NTP & PPS, part 2 ;)
Rob writes: > Harlan Stenn wrote: > > Paul writes: > >> On Tue, Dec 16, 2014 at 10:10 AM, Martin Burnicki < > >> martin.burni...@meinberg.de> wrote: > >> > >> > OK, but what is the problem in using these IOCTLs directly from within > >> > ntpd, via wrapper functions or directly? Several refclock drivers do so. > >> > >> You'll have to ask Harlan. > > > > Useful discussion and patches would be nice. > > The code does not have to be patched because it already works, you only > need to move the timepps.h file into the distributed source. It provides > the wrapper functions between the existing refclocks and the IOCTLs. Since you don't see my POV and I've looked at yours several times and always see how doing what you suggest will cause horrible headaches and problem, something else will have to shift. H ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] NTP & PPS, part 2 ;)
Harlan Stenn wrote: > Rob writes: >> Harlan Stenn wrote: >> > Paul writes: >> >> On Tue, Dec 16, 2014 at 10:10 AM, Martin Burnicki < >> >> martin.burni...@meinberg.de> wrote: >> >> >> >> > OK, but what is the problem in using these IOCTLs directly from within >> >> > ntpd, via wrapper functions or directly? Several refclock drivers do so. >> >> >> >> You'll have to ask Harlan. >> > >> > Useful discussion and patches would be nice. >> >> The code does not have to be patched because it already works, you only >> need to move the timepps.h file into the distributed source. It provides >> the wrapper functions between the existing refclocks and the IOCTLs. > > Since you don't see my POV and I've looked at yours several times and > always see how doing what you suggest will cause horrible headaches and > problem, something else will have to shift. When you really think that doing the above will cause headaches, then the direct use of IOCTLs will do even more so. With my solution you can at least update the timepps.h file in the case there would be a change in the IOCTLs, when you access those directly you will have to solve the problem yourself. ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Number of Stratum 1 & Stratum 2 Peers
Rob wrote: Martin Burnicki wrote: Imagine you set up an event for April 2015 today, but you just don't know if DST will be in effect at that time, or not, just because the politicians haven't made the decision today. How will you handle this? It may not be helpful if you get updated DST rules the day *after* the scheduled event. On the other hand, when the event time approaches and you have the chance to get a DST rule update before the event time everything may be fine, if a decision is taken either to keep the UTC time *or* the local time for this event. Since UTC is actually used in most computers to keep the system time and schedule events based on an accurate UTC time it may not be helpful to do this with post-processing. It depends on what kind of event it is. When it is a "human event" like a meeting, it is customary to plan these events in local time. "It depends" even more. The tzdist folks are planning to use a reference to a specific time zone instead of the TZ rules which have been in effect when the event was created. I've recently posted some consideration on the tzdst mailing list: http://www.ietf.org/mail-archive/web/tzdist/current/msg01063.html So they should be stored in local time with a specified timezone. What UTC time this is will depend on the local time rules at the actual event time. It will still be tricky when you plan a meeting at 02:30 on the sunday of the DST change from summer->winter time. You cannot add a DST flag with the event because you cannot know if there will be a DST change at that time. Of course that is why the DST change is done at a moment where such problems are unlikely, and not for example at 12:00 on the first day of the month. That's why IMO they should be stored in UTC time (or even TAI, if that was possible), and only presented to the user in preferred local time zone. I've posted some thoughts for the case that the TZ/DST rules change afterwards in my email behind the link above. Some calendering software gets this wrong, yes. E.g. by converting the specified local time to UTC time and storing that. This has caused problems in the past for well-known calendering software, I don't know if that has been fixed. For "technical" events it may be different, and storing them at the UTC time could be better. But scheduling software (Windows Scheduler, Unix cron) usually uses local time as well. Unfortunately it seems to be common practice today to write software in a way it is quickly ready and works in most cases, without thinking about different cases or what happens if anything is not as usually expected. See the discussion about the expiration date of the leap second table. It is not hard to implement this, and if it is supported it allows software to be written in a user-friendly way which allows to generate a warning to users or admin saying that interaction may be required, e.g. information (leap second tables, DST rules) need to be updated. Of course you can say a good admin has to take care of this, but I'm sure an admin would be happy if he received a reminder just in case he doesn't remember a specific task. And of course you can say this works perfectly without expiration date if every tiny client gets the current data via an SSL connection directly from NIST, IERS, or IANA, but if you *do* provide an expiration date both this works nevertheless in the same way, but allows for extended usage which is more user-friendly and thus avoids potential risks. Martin ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Number of Stratum 1 & Stratum 2 Peers
Rob wrote: Martin Burnicki wrote: IMO the GPS system designers have made quite a number of wise decisions, e.g. letting the GPS time simply increase monotonically, which is, from a technical/usage point of view, similar to TAI. That decision was wise. The decision to express date in weeks and put it in a 10-bit field was not so wise. Oh well. Only the 10 bit field wasn't. However, for a pure navigation system this doesn't matter at all. Only if you need the current legal date and time the epoch matters. With regard to this, using only a truncated 8 bit week number to specify the week of a leap second event is also not the best solution. It can only handle a +/-127 week interval, and expanding this to a full date is ambiguous if there was no leap second event for a period exceeding this interval. Martin ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Default total number of servers NTP wants to have when using pool .....
On Wed, Dec 17, 2014 at 12:04:04PM +, Harlan Stenn wrote: > I'd love to see discussion about "what should the default number of > servers queried be for the 'pool' directive?" The "How do I use pool.ntp.org?" page [1] is pretty clear, quoting: Be friendly. Many servers are provided by volunteers, and almost all time servers are really file or mail or webservers which just happen to also run ntp. So don't use more than four time servers in your configuration, and don't play tricks with burst or minpoll - all you will gain is extra load on the volunteer time servers. > There is clearly a tradeoff, and I'm inclined to say that "between 5 and > 9" is probably a good number. Ok, but examples of ntpd configuration using pool.ntp.org should follow their policy. Maybe you can convince them to change it. Do you think the servers are ready to handle twice as many clients? [1] http://www.pool.ntp.org/en/use.html -- Miroslav Lichvar ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Default total number of servers NTP wants to have when using pool .....
On Wed, Dec 17, 2014 at 7:04 AM, Harlan Stenn wrote: > I'd love to see discussion about "what should the default number of > servers queried be for the 'pool' directive?" > I don't think it matters. Properly configured systems and sub-nets will have little impact and poorly configured systems can have significant impact regardless of maxclocks. ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Number of Stratum 1 & Stratum 2 Peers
Martin Burnicki wrote: > Rob wrote: >> Martin Burnicki wrote: >>> IMO the GPS system designers have made quite a number of wise decisions, >>> e.g. letting the GPS time simply increase monotonically, which is, from >>> a technical/usage point of view, similar to TAI. >> >> That decision was wise. The decision to express date in weeks and put >> it in a 10-bit field was not so wise. Oh well. > > Only the 10 bit field wasn't. However, for a pure navigation system this > doesn't matter at all. Only if you need the current legal date and time > the epoch matters. Sure, but when your require the system only for navigation and not to provide time there is no reason whatsoever to sync it with any known time standard or even to tick in units of a second. Apparently the application of providing UTC time was considered in the design phase, but it was never thought the system would be deployed and would remain operational and be widely universally used for more than 19 years. In this regard, it is similar to the Internet. ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Number of Stratum 1 & Stratum 2 Peers
Harlan Stenn wrote: Martin Burnicki writes: Rob wrote: Unfortunately, the same mechanism isn't used for leap seconds. There would be no problem at all when the system time ticked in TAI and the addition of the leap seconds is done via some rule table similar to the local time rules. ntpd would not even be involved in the problem. Also agreed. However, actually ntpd expects UTC, not TAI. One of the things I'm thinking about for NTP5 is systems that choose to run in TAI and not UTC. This is a natural tie-in with NTF's General Timestamp API, as we need to know what timescale the remote system is using. Of course. However, you would anyway need the current rules or algorithms required to do the conversion. This woud also handle the case of maintaining sync with a system that chooses to implement Google's leap smear, Do you know the *exact* algorithm how Google smears a leap second? If you don't know it there will be differences in the time computed by standard ntpd v5 and Google's. ;-) Martin ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Number of Stratum 1 & Stratum 2 Peers
On 2014-12-16 10:28, Phil W Lee wrote: Martin Burnicki considered Tue, 16 Dec 2014 12:48:57 +0100 the perfect time to write: Brian Inglis wrote: It would be interesting to know what percentage of the pool servers even use a leapseconds file, and how many of those have a valid copy. I am certain that very few clients use a leapseconds file. OTOH the timezone/zoneinfo package uses its own leapseconds file (for "right" time - now zoneinfo-leaps), and distributes that and the original, a script that checks and converts it to their own format, and utilities that use it. As far as I can see the the leap second file shipped with the tzdata package is generated automatically from a leap second file in NIST format. However, there is an important limitation: the tzdata version of the leap second file is missing an expiration date, so even if a program like ntpd could use this file directly it would never know if no more leap second has been scheduled after the last one mentioned in this file, or if the file just hasn't been updated recently enough. If it compared the file it has with the one available from tzdata, it could see if there was a difference - it's not exactly huge. Even better would be if a checksum or version number was available as well, so that could just be compared to the current one, and a new one only downloaded if required. The HTTP modified header does that well if provided by the web server for a file - it is for most files on most web servers, also for FTP files. All my network update scripts use wget -N to timestamp files and check the modified header to see if an update needs downloaded. Update scripts can then check to see if a file timestamp, and internal expiry date, release date, or version number has changed, and checksums are correct, to decide whether to apply the update, or if it can be skipped. -- Take care. Thanks, Brian Inglis ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] NTP & PPS, part 2 ;)
On 17/12/2014 08:52, Martin Burnicki wrote: [] This is just a subset of the information you get from "ntpq -c rv", e.g.: associd=0 status=0615 leap_none, sync_ntp, 1 event, clock_sync, version="ntpd 4.2.6p5@1.2349-o Jul 30 11:55:08 (UTC+02:00) 2012 (2)", processor="x86", system="Windows", leap=00, stratum=2, precision=-10, rootdelay=0.015, rootdisp=4.447, refid=172.16.3.9, reftime=d83bc128.df638c2d Wed, Dec 17 2014 9:45:28.872, clock=d83bc157.26a94acd Wed, Dec 17 2014 9:46:15.151, peer=29231, tc=6, mintc=3, offset=0.171, frequency=500.000, sys_jitter=0.000, clk_jitter=0.985, clk_wander=0.003 telling on which system ntpd is running, *not* on which system it has been built. Martin That fails here, then, as I'm running on 64-bit Windows and seeing x86 as the processor. Oh, and NTP was built on a 64-bit system as a 32-bit application! Yes, I understand why this is -- Cheers, David Web: http://www.satsignal.eu ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
[ntp:questions] linux kernel pps support on raspberry pi
This is about receiving a pps signal via one of the gpio pins of the
raspberry pi and then feeding it to ntp.
Did a test for a while: every 5 minutes I would look at the output of
ntpq -c pe -n for jitter of the pps source.
This pps source was "measured" either in the kernel (this newly added
gpio pps support in the raspberry pi) or in user space using my own
rpi_gpio_ntp.
kernel pps interrupt handling
-
root@detijd:/etc# while true ; do echo `date +%s` `ntpq -c pe -n | grep
127.127.22.0` ; sleep 300 ; done | tee -a /pps.log
root@detijd:/etc# cat /pps.log | awk '{ print $11; }' | awk 'BEGIN { max = -1;
min = 1000; } { if ($1 > max) { max=$1; } if ($1 < min) { min=$1; } t+=$1; n++;
} END { print min, max, t/n, n; }'
0.001 0.124 0.0131945 797
This is minimum, maximum (yes that's 124!) and average. And I took 797
samples which is almost 3 days.
average temperature of soc: 49.7 with stddev of 1.87
userspace
-
note: I looked at ntpq every 30 seconds(!)
root@hetlicht:/var/log/ntpstats# cat /rpn.log | awk '{ print $11; }' | awk
'BEGIN { max = -1; min = 1000; } { if ($1 > max) { max=$1; } if ($1 < min) {
min=$1; } t+=$1; n++; } END { print min, max, t/n, n; }'
0.001 0.101 0.00955351 27797
This is over 9,6 days.
average temperature of soc: 51,1 with stddev of 4.01
TL;DR:
- kernel : 13.2us jitter on average
- userspace: 9.6us jitter on average
Folkert van Heusden
--
--
www.smartwinning.info
___
questions mailing list
questions@lists.ntp.org
http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] linux kernel pps support on raspberry pi
Is the Rpi overclocked? When overclocked it will dynamically change the CPU frequency. You could potentially underclock it or heatsink the SoC to keep the CPU frequency from changing. Also, which version of the RPi? A, B or B-plus? On 2014-12-17 12:22, folkert wrote: > This is about receiving a pps signal via one of the gpio pins of the > raspberry pi and then feeding it to ntp. > > Did a test for a while: every 5 minutes I would look at the output of > ntpq -c pe -n for jitter of the pps source. > This pps source was "measured" either in the kernel (this newly added > gpio pps support in the raspberry pi) or in user space using my own > rpi_gpio_ntp. > > > TL;DR: > - kernel : 13.2us jitter on average > - userspace: 9.6us jitter on average > > > Folkert van Heusden > ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] NTP & PPS, part 2 ;)
Rob writes: > Harlan Stenn wrote: > > Rob writes: > >> Harlan Stenn wrote: > >> > Paul writes: > >> >> On Tue, Dec 16, 2014 at 10:10 AM, Martin Burnicki < > >> >> martin.burni...@meinberg.de> wrote: > >> >> > >> >> > OK, but what is the problem in using these IOCTLs directly from withi > n > >> >> > ntpd, via wrapper functions or directly? Several refclock drivers do > so. > >> >> > >> >> You'll have to ask Harlan. > >> > > >> > Useful discussion and patches would be nice. > >> > >> The code does not have to be patched because it already works, you only > >> need to move the timepps.h file into the distributed source. It provides > >> the wrapper functions between the existing refclocks and the IOCTLs. > > > > Since you don't see my POV and I've looked at yours several times and > > always see how doing what you suggest will cause horrible headaches and > > problem, something else will have to shift. > > When you really think that doing the above will cause headaches, then > the direct use of IOCTLs will do even more so. With my solution you > can at least update the timepps.h file in the case there would be a > change in the IOCTLs, when you access those directly you will have to > solve the problem yourself. Please post a copy of this timepps.h file that will work for all versions of all OSes that are running NTP. H ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Default total number of servers NTP wants to have when using pool .....
Miroslav Lichvar writes: > On Wed, Dec 17, 2014 at 12:04:04PM +, Harlan Stenn wrote: > > I'd love to see discussion about "what should the default number of > > servers queried be for the 'pool' directive?" > > The "How do I use pool.ntp.org?" page [1] is pretty clear, quoting: > > Be friendly. Many servers are provided by volunteers, and almost > all time servers are really file or mail or webservers which just > happen to also run ntp. So don't use more than four time servers > in your configuration, and don't play tricks with burst or > minpoll - all you will gain is extra load on the volunteer time > servers. > > > There is clearly a tradeoff, and I'm inclined to say that "between 5 and > > 9" is probably a good number. > > Ok, but examples of ntpd configuration using pool.ntp.org should follow > their policy. Maybe you can convince them to change it. Do you think > the servers are ready to handle twice as many clients? > > [1] http://www.pool.ntp.org/en/use.html What works best for a running instance of ntpd is not always going to be what works best for the pool servers. Choosing the right balance is something that happens "now" and the balance point will change over time. The ntp.conf generator project NTF wants to work on can help with that. As soon as NTF gets the resources to move forward with that, it will happen. With decent GPS pucks costing http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Number of Stratum 1 & Stratum 2 Peers
Martin Burnicki writes: > Harlan Stenn wrote: > > Martin Burnicki writes: > >> Rob wrote: > >>> Unfortunately, the same mechanism isn't used for leap seconds. > >>> There would be no problem at all when the system time ticked in TAI > >>> and the addition of the leap seconds is done via some rule table similar > >>> to the local time rules. ntpd would not even be involved in the problem. > >> > >> Also agreed. However, actually ntpd expects UTC, not TAI. > > > > One of the things I'm thinking about for NTP5 is systems that choose to > > run in TAI and not UTC. This is a natural tie-in with NTF's General > > Timestamp API, as we need to know what timescale the remote system is > > using. > > Of course. However, you would anyway need the current rules or > algorithms required to do the conversion. > > > This woud also handle the case of maintaining sync with a system that > > chooses to implement Google's leap smear, > > Do you know the *exact* algorithm how Google smears a leap second? If > you don't know it there will be differences in the time computed by > standard ntpd v5 and Google's. ;-) I know what they've published. If we assign unique "leap method numbers" and folks actually use these properly, at worst we would say "Don't sync with Google Leap Smear machines on the day a leap second is being processed". If a system is using an older NTP it will ID as v4, and any ntp5 instances will be able to "know" to be careful. -- Harlan Stenn http://networktimefoundation.org - be a member! ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
