Re: [ntp:questions] NIST vs. pool.ntp.org ?
Il 03/27/2013 10:24 PM, unruh ha scritto: You do NOT want to hard code anything into your program. That is extremely bad form, unless that address is one controlled by you. Indeed. Robert, please see: http://en.wikipedia.org/wiki/NTP_server_misuse_and_abuse and in particular: http://en.wikipedia.org/wiki/NTP_server_misuse_and_abuse#D-Link_and_Poul-Henning_Kamp a notable case among already notable cases... Ciao -- bronto ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] NIST vs. pool.ntp.org ?
Robert Scott no-one@notreal.invalid wrote: To achieve 11 ppm accuracy in frequency I need to have a calibration time interval that is about 90,000 times as long as the timestamp uncertainty. If the timestamp uncertainty is, say, 100 msec., the calibration time period needs to be at least 2.5 hours. That's where my figure of 3 hours comes from. I don't think it will difficult for a user of my app to perform this calibration. All he has to do is to ensure Internet connectivity is turned on (it could be cell or wi-fi), hit the calibrate button in my app, and leave the phone on charge and go to bed. To achieve an uncertainty below 100ms over connections like that you will need to do some clever programming where you get multiple time stamps, measure the roundtriptime on each of them, and discard outliers before you calculate an average. The kind of thing that ntpd already does. Radio protocols often use polling to grant the permission to the mobiles to send, and adaptive polling intervals depending on the recent traffic. (or they drop clients from the polling list when there is no traffic, offering some method for them to join again) The result is that when you start traffic on an otherwise idle client, you will observe roundtrip intervals like this: time=531 ms time=500 ms time=540 ms time=874 ms time=99.7 ms time=101 ms time=96.3 ms time=97.3 ms time=101 ms time=96.9 ms time=102 ms time=102 ms time=97.4 ms time=102 ms time=103 ms in this example, the first four exchanges were with the device still in idle mode and only then the cell network (UMTS in this case) decided to step down the polling interval on that client, and from the fifth one the roundtriptime is pretty consistent. you will need to discard those first 4 samples and then calculate an average over a couple more. however, that 4 is not a constant! it depends on parameters and circumstances beyond your control. so you will need to write an adaptive algorithm that recognizes what is happening here, and send the queries quickly enough (I would say at least two per second, maybe 4) to force the active user algorithm to kick in. even then, I have seen links (e.g. over WiFi) where a stable state is never reached and varying roundtriptimes between nearly zero and about 200ms are seen no matter how often and how frequently you ping over them. ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] NIST vs. pool.ntp.org ?
On 03/27/2013 10:45 PM, David Woolley wrote: Robert Scott wrote: I am confused about the proper usage of pool.ntp.org and NIST. pool.ntp.org seems to be a collection of private sector time servers offered for all to use, but with registration expected for regular The pool system has no provision for enforcing registration. It wouldn't make sense to hand out a random server address if most of them then refused to serve you because you hadn't registered. users. And NIST has a government run set of time servers. Neither group (NIST or pool.ntp.org) seems to include or referece the other. I would hope all the pool servers ultimately reference their national equivalent of NIST and therefore what becomes, after the fact, UTC. I think you will find that Navstar (GPS) and WWV times are traceable to NIST. Yes and no. GPS is traceable to USNO. USNO and NIST have traceability between each other within the BIPM framework. MSF times are traceable to NPL. NPL is traceable to both USNO and NIST within the BIPM framework. Are they in competition? Who normally uses the NIST servers and who uses pool.ntp.org? The open NIST servers are heavily overloaded, so probably don't serve the highest quality time, but they are likely to be around for a long time. I would setup a local server under your control. It will help both from debugging and noise perspective. Cheers, Magnus ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] NIST vs. pool.ntp.org ?
Hi again Robert, On 03/28/2013 04:22 AM, Robert Scott wrote: On Thu, 28 Mar 2013 02:50:17 GMT, unruhun...@invalid.ca wrote: You really should read my posts before responding. No, I do not intend to hard-code NIST or any other server. I never said I wanted to. No, the app is not intended for all musicians. It is intended for professional piano tuners only. I sell about one per day. And I never said the pool would not be good enough for my needs. I only asked about the relative benefits of the pool vs. NIST, which E-mail sent...Blacklists answered very nicely. There is no real benefit in using either, rather you should use the mix of servers which gives you good confidence in removing false-tickers as well as good precision due to use of short distances. Look at the NTP code and book, as many of the filtering steps aims at removing noise which polute the time and frequency errors. Do the to-way time-transfer. Cheers, Magnus ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] NIST vs. pool.ntp.org ?
On 28 Mar 2013 09:57:37 GMT, Rob nom...@example.com wrote: To achieve an uncertainty below 100ms over connections like that you will need to do some clever programming where you get multiple time stamps, measure the roundtriptime on each of them, and discard outliers before you calculate an average. The kind of thing that ntpd already does. If I could use a packaged implementation of NTP I would. But I don't have that option. If I understand correctly, packaged implementations of NTP are designed specifically for setting the system time. I cannot impose that effect on users of my app. My app must be stand-alone and not attempt to affect the system time. So I will have to take the algorithms and basic ideas of NTP and re-implement them in a stand-along fashion. In any case the language of implementation will have to depend on the target: Java for Android devices, Objective-C for iPhones. Radio protocols often use polling to grant the permission to the mobiles to send, and adaptive polling intervals depending on the recent traffic. (or they drop clients from the polling list when there is no traffic, offering some method for them to join again) The result is that when you start traffic on an otherwise idle client, you will observe roundtrip intervals like this: time=531 ms time=500 ms time=540 ms time=874 ms time=99.7 ms time=101 ms time=96.3 ms time=97.3 ms time=101 ms time=96.9 ms time=102 ms time=102 ms time=97.4 ms time=102 ms time=103 ms Are these exchanges all with the same Time Server? I thought good citizen use of these Time Servers required at least 4 seconds inbetween queries. Or does the start-up polling time penalty only once even if every query is to a different Time Server? in this example, the first four exchanges were with the device still in idle mode and only then the cell network (UMTS in this case) decided to step down the polling interval on that client, and from the fifth one the roundtriptime is pretty consistent. you will need to discard those first 4 samples and then calculate an average over a couple more. however, that 4 is not a constant! it depends on parameters and circumstances beyond your control. so you will need to write an adaptive algorithm that recognizes what is happening here, and send the queries quickly enough (I would say at least two per second, maybe 4) to force the active user algorithm to kick in. even then, I have seen links (e.g. over WiFi) where a stable state is never reached and varying roundtriptimes between nearly zero and about 200ms are seen no matter how often and how frequently you ping over them. In those cases how asymmetric is the polling delay likely to be if I just take the midpoint of the polling interval, just like NTP? I realize that it is possible to be extremely assymetric, but in practice? Robert Scott Hopkins, MN ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] NIST vs. pool.ntp.org ?
Robert Scott no-one@notreal.invalid wrote: On 28 Mar 2013 09:57:37 GMT, Rob nom...@example.com wrote: To achieve an uncertainty below 100ms over connections like that you will need to do some clever programming where you get multiple time stamps, measure the roundtriptime on each of them, and discard outliers before you calculate an average. The kind of thing that ntpd already does. If I could use a packaged implementation of NTP I would. But I don't have that option. I understand that, I only want you to know that you need to implement a clever filter just like in ntpd or else you will have a problem. Are these exchanges all with the same Time Server? I thought good citizen use of these Time Servers required at least 4 seconds inbetween queries. Or does the start-up polling time penalty only once even if every query is to a different Time Server? That is a complicating factor. You need to cause some traffic to kick the link into faster action but you don't want to cause trouble with the Time Server admin (who may have set an access rule that blocks you when you send a burst of requests). You can instead ping something that you know responds to pings, e.g. send 10 requests at .25 second intervals, and then immediately request the time from a timeserver. The link does not look at what system you communicate with, it only checks if there is a certain number of packets per second to see if you are active. even then, I have seen links (e.g. over WiFi) where a stable state is never reached and varying roundtriptimes between nearly zero and about 200ms are seen no matter how often and how frequently you ping over them. In those cases how asymmetric is the polling delay likely to be if I just take the midpoint of the polling interval, just like NTP? I realize that it is possible to be extremely assymetric, but in practice? Very asymmetric. The access point can always transmit when it likes, the clients have to wait for the access point to tell them that they can have a go. It varies widely between implementations. It can be like the behaviour shown for UMTS, or it can be that the access point polls at some fixed interval (that you aren't synchronous to) so you will see variation between 0 and that interval, only in the direction client-server. Maybe for your application it would be wise to check the roundtriptime only to see if it has already fallen from some initial high value, but not to use it in the calculation of real time (i.e. don't take a midpoint between request and reply). That is because excessive and variable delays are more likely to occur on the uplink than on the downlink. ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] NIST vs. pool.ntp.org ?
On 28 Mar 2013 13:37:50 GMT, Rob nom...@example.com wrote: Maybe for your application it would be wise to check the roundtriptime only to see if it has already fallen from some initial high value, but not to use it in the calculation of real time (i.e. don't take a midpoint between request and reply). That is because excessive and variable delays are more likely to occur on the uplink than on the downlink. I think what I may do is make the most conservative assumption about the assymetry, i.e. that it is as bad as possible. Then calculate the precision based on the round-trip time. If the round-trip time is small enough then I don't care about the asymmetry. Since the overall calibration period is under my control, and is only limited by the time the user leaves his phone on charge overnight, I could potentially wait for 7 or 8 hours before taking my final time-stamp reading. That will relax the time-stamp precision to something more like 280 msec. Robert Scott Hopkins, MN ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] NIST vs. pool.ntp.org ?
On Thu, Mar 28, 2013 at 5:57 AM, Rob nom...@example.com wrote: so you will need to write an adaptive algorithm that recognizes what is happening here, and send the queries quickly enough (I would say at least two per second, maybe 4) to force the active user algorithm to kick in. If he sends requests too fast when the latency is so high, isn't there a chance they'll just get combined due to Nagle which will throw off his calculations even more? ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] NIST vs. pool.ntp.org ?
Trevor Woerner twoer...@gmail.com wrote: On Thu, Mar 28, 2013 at 5:57 AM, Rob nom...@example.com wrote: so you will need to write an adaptive algorithm that recognizes what is happening here, and send the queries quickly enough (I would say at least two per second, maybe 4) to force the active user algorithm to kick in. If he sends requests too fast when the latency is so high, isn't there a chance they'll just get combined due to Nagle which will throw off his calculations even more? This is UDP, not TCP. Of course there is not much point in shotgunning requests when there is no reply. First attempt could be to send a new request only after the first reply has been received, but I am not sure this will always guarantee the radio link goes to active mode. Maybe send 3 requests at .25 or .5 second intervals ant then wait until at least one reply comes back. With GPRS it sometimes takes several seconds for the first replies to come back. ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] NIST vs. pool.ntp.org ?
A report on UTC(NIST) - UTC(USNO) can be found at http://tf.boulder.nist.gov/general/pdf/2663.pdf On Thu, Mar 28, 2013 at 7:22 AM, Magnus Danielson mag...@rubidium.dyndns.org wrote: On 03/27/2013 10:45 PM, David Woolley wrote: Robert Scott wrote: I am confused about the proper usage of pool.ntp.org and NIST. pool.ntp.org seems to be a collection of private sector time servers offered for all to use, but with registration expected for regular The pool system has no provision for enforcing registration. It wouldn't make sense to hand out a random server address if most of them then refused to serve you because you hadn't registered. users. And NIST has a government run set of time servers. Neither group (NIST or pool.ntp.org) seems to include or referece the other. I would hope all the pool servers ultimately reference their national equivalent of NIST and therefore what becomes, after the fact, UTC. I think you will find that Navstar (GPS) and WWV times are traceable to NIST. Yes and no. GPS is traceable to USNO. USNO and NIST have traceability between each other within the BIPM framework. MSF times are traceable to NPL. NPL is traceable to both USNO and NIST within the BIPM framework. Are they in competition? Who normally uses the NIST servers and who uses pool.ntp.org? The open NIST servers are heavily overloaded, so probably don't serve the highest quality time, but they are likely to be around for a long time. I would setup a local server under your control. It will help both from debugging and noise perspective. Cheers, Magnus __**_ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/**questionshttp://lists.ntp.org/listinfo/questions ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] NIST vs. pool.ntp.org ?
On Thu, Mar 28, 2013 at 5:57 AM, Rob nom...@example.com wrote: so you will need to write an adaptive algorithm that recognizes what is happening here, and send the queries quickly enough (I would say at least two per second, maybe 4) to force the active user algorithm to kick in. Sending at a rate of more than 1 packet every 2 seconds or more than 8 packets per 64 seconds will trigger KOD responses. H ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] NIST vs. pool.ntp.org ?
On Thu, 28 Mar 2013 21:21:55 GMT, Harlan Stenn st...@ntp.org wrote: On Thu, Mar 28, 2013 at 5:57 AM, Rob nom...@example.com wrote: so you will need to write an adaptive algorithm that recognizes what is happening here, and send the queries quickly enough (I would say at least two per second, maybe 4) to force the active user algorithm to kick in. Sending at a rate of more than 1 packet every 2 seconds or more than 8 packets per 64 seconds will trigger KOD responses. Perhaps these wake-up packets could be some other UDP packet, like a Ping, and not to some time server, but to some other internet address, like www.microsoft.com (not really!). If I understand Rob's suggestion, all that is needed is some kind of activity to wake up the radio/internet channel, not necessarily NTP activity. Robert Scott Hopkins, MN ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] NIST vs. pool.ntp.org ?
Robert Scott wrote: If I could use a packaged implementation of NTP I would. But I don't have that option. If I understand correctly, packaged implementations of NTP are designed specifically for setting the system time. I The interface to the OS clock is fairly well abstracted for portability reasons. If you forego the kernel time discipline, you should be able to modify an offset from system time, rather than the system time itself, although the fact that ntpd really controls frequency may be a bit of a problem, as you can only really fake offset. ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] NIST vs. pool.ntp.org ?
On 2013-03-28, Robert Scott no-one@notreal.invalid wrote: On 28 Mar 2013 09:57:37 GMT, Rob nom...@example.com wrote: To achieve an uncertainty below 100ms over connections like that you will need to do some clever programming where you get multiple time stamps, measure the roundtriptime on each of them, and discard outliers before you calculate an average. The kind of thing that ntpd already does. If I could use a packaged implementation of NTP I would. But I don't have that option. If I understand correctly, packaged implementations of NTP are designed specifically for setting the system time. I cannot impose that effect on users of my app. My app must be stand-alone and not attempt to affect the system time. So I will have to take the algorithms and basic ideas of NTP and re-implement them in a stand-along fashion. In any case the language of implementation will have to depend on the target: Java for Android devices, Objective-C for iPhones. Well, you could always tell ntpd not to use the outside sources to set the time. Radio protocols often use polling to grant the permission to the mobiles to send, and adaptive polling intervals depending on the recent traffic. (or they drop clients from the polling list when there is no traffic, offering some method for them to join again) The result is that when you start traffic on an otherwise idle client, you will observe roundtrip intervals like this: time=531 ms time=500 ms time=540 ms time=874 ms time=99.7 ms time=101 ms time=96.3 ms time=97.3 ms time=101 ms time=96.9 ms time=102 ms time=102 ms time=97.4 ms time=102 ms time=103 ms Are these exchanges all with the same Time Server? I thought good citizen use of these Time Servers required at least 4 seconds inbetween queries. Or does the start-up polling time penalty only once even if every query is to a different Time Server? in this example, the first four exchanges were with the device still in idle mode and only then the cell network (UMTS in this case) decided to step down the polling interval on that client, and from the fifth one the roundtriptime is pretty consistent. you will need to discard those first 4 samples and then calculate an average over a couple more. however, that 4 is not a constant! it depends on parameters and circumstances beyond your control. so you will need to write an adaptive algorithm that recognizes what is happening here, and send the queries quickly enough (I would say at least two per second, maybe 4) to force the active user algorithm to kick in. even then, I have seen links (e.g. over WiFi) where a stable state is never reached and varying roundtriptimes between nearly zero and about 200ms are seen no matter how often and how frequently you ping over them. In those cases how asymmetric is the polling delay likely to be if I just take the midpoint of the polling interval, just like NTP? I realize that it is possible to be extremely assymetric, but in practice? Robert Scott Hopkins, MN ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
