Re: [ntp:questions] Polling interval in FreeBSD vs. Windows
Mike S wrote: At 10:35 PM 1/18/2011, Dave Hart wrote... On Wed, Jan 19, 2011 at 02:47 UTC, Mike S mi...@flatsurface.com wrote: At 02:18 PM 1/18/2011, unruh wrote... Especially as ntpd uses only one of every 8 polls, Is this relatively recent behavior? No. See the Clock filter algorithm slide 13 in: http://www.eecis.udel.edu/~mills/database/brief/arch/arch.pdf That's dated Jul2007. My point of reference is whatever version of NTP was used in Debian Lenny (4.2.4p4, from Aug2007, it seems) which is what I was running before changing to Squeeze (4.2.6p2) a couple of months ago. Perhaps a change was made about that time? When 4.2.4p4 was published as a patch release for -stable, the current -dev version was 4.2.5p74/p75. See: http://ntp.bkbits.net:8080/ntp-dev/?PAGE=tags Please note Dave Mill's publications usually refer to the -dev branch which is current at the time of publication. 4.2.5 started in Dec 2006, so I expect the change has been introduced sometime after Dec 2006 an not necessarily around Aug 2007. Martin -- Martin Burnicki Meinberg Funkuhren Bad Pyrmont Germany ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Polling interval in FreeBSD vs. Windows
On Tue, Jan 18, 2011 at 09:48:08PM +, David Woolley wrote:
That was my point. Unruh's main issue is that, on modern LANs, the
dominant low frequency error is in the local clock, rather than the
measurements.
That's the theory behind NTP.
It's more complicated. I don't think the current version ages the
samples,
It still does, see clock_filter() in ntp_proto.c:
dtemp = clock_phi * (current_time - peer-update);
I
think the filter will take out more like 7 in 8 for gaussian input,
but the expected input pattern isn't actually gaussian, either.
I think the delay is assumed to be exponentially distributed (and
that's what I use in my simulations). It would be interesting to
analyse real data from rawstats.
Also, to answer the question whether PLL has a good lock, you can use
the runs tests. Run this command on your loopstats and we'll see if
the offset is random or not.
awk '{ n++; r += $3 * l 0; l = $3 } END { print r / n }'
--
Miroslav Lichvar
___
questions mailing list
questions@lists.ntp.org
http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Polling interval in FreeBSD vs. Windows
At 10:35 PM 1/18/2011, Dave Hart wrote... On Wed, Jan 19, 2011 at 02:47 UTC, Mike S mi...@flatsurface.com wrote: At 02:18 PM 1/18/2011, unruh wrote... Especially as ntpd uses only one of every 8 polls, Is this relatively recent behavior? No. See the Clock filter algorithm slide 13 in: http://www.eecis.udel.edu/~mills/database/brief/arch/arch.pdf That's dated Jul2007. My point of reference is whatever version of NTP was used in Debian Lenny (4.2.4p4, from Aug2007, it seems) which is what I was running before changing to Squeeze (4.2.6p2) a couple of months ago. Perhaps a change was made about that time? Maybe I'm just getting old and feeble, but I swear ntpq -p used to update the offset with almost every poll - I would watch it after making a change to get a feel for how the offset and drift were adjusting (watch ntpq -p; ntptime). Or, perhaps, NTP used to report the most recently _measured_ offset/jitter, and was changed to reporting the most recently _used_? Another change I've noticed with very recent versions (maybe a 4.2.7p?), I didn't keep track) is that the polling interval seems to add a random time to what's expected. If watching ntpq -p, I will often see when count significantly beyond the current poll value. It doesn't indicate a missing response when it finally does seem to poll. ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Polling interval in FreeBSD vs. Windows
Dave Hart wrote: On Tue, Jan 18, 2011 at 19:18 PM, unruhun...@wormhole.physics.ubc.ca wrote: On 2011-01-18, David Woolleydavid@ex.djwhome.demon.invalid wrote: wander, and you set a very fast poll, you could get low offsets but a high error, because the real error is in the wander.  Increasing the No. a fast poll also corrects wander faster. ntpd is horribly slow at correcting wander, but doing it quickly corrects it faster. I would agree if you said perceived wander with measurement-induced error. In other words, while turning the slew knob faster reduces your offsets faster, doing so risks following measurement noise rather than the underlying trend. What it does not do  is to give a good estimate of wander. The drift correction flops around much more with faster polls, because of the short baseline. That does not matter if you have constant connectivity, but becomes important if you loose connectivity for any time period. It matters to me. I want my frequency slewing to ignore measurement noise, not react quickly to it. This may be complicated in that I'm not sure that the loop time constant is actually clamped by the poll rate, so it is possible that setting a high rate over samples without changing the overall behaviour.  Maybe it is clamped by maxpoll? No, it is determined by the number of readings, not by time. More readings in a given time means shorter time constant. Fewer readeings, longer. ntpd throws away 7 out of 8 readings, so the actual poll interval of used reading is 8 times as long as the set poll interval, and the time constant is about 5 times that if I remember correctly. You do not understand NTP's clock filter. As each new sample is pushed in, only one of the prior 8 samples is used for the moment, but each sample has 8 chances to be that favorite. That is not equivalent to throwing away 7 out of 8 readings on my planet. There's also the fact that for refclocks ntpd will measure/save the PPS signal that arrives every second, a larger poll interval just means that there are more samples to choose from. I.e. with minpoll 4 my Oncore or Garmin refclocks will deliver 16 samples into the core engine for every poll period. Terje -- - Terje.Mathisen at tmsw.no almost all programming can be viewed as an exercise in caching ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Polling interval in FreeBSD vs. Windows
Miroslav Lichvar wrote: On Tue, Jan 18, 2011 at 09:48:08PM +, David Woolley wrote: That was my point. Unruh's main issue is that, on modern LANs, the dominant low frequency error is in the local clock, rather than the measurements. That's the theory behind NTP. I should probably have said mid frequency. The specific problem with NTP that he complains about is that it sets it loop time constant too long (or more precisely fails to adapt it down fast enough) such that mid-frequency clock errors are removed from the error measurement by the loop filter and the time drifts badly in response to them. This is the behaviour you would want in respect of mid-frequency network delay variations, e.g. the sort of problem I was describing where the delay jumps up for a couple of ours around 1pm when everyone is browsing during their lunch break. (As the temperature variations (and network delay variations) often have squarish shapes, they actually have a broad spectrum.) ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Polling interval in FreeBSD vs. Windows
unruh wrote: ?? Surely what you want is that your computer's clock shows the correct time. If your computer disciplines the clock to have the true rate (1 sec of computer time equals one second of UTC time) but was 10 hours off, I think most people would count that as a failure of ntp. Not sure what you mean with You cannot do that in an unqualified way. The actual error, when locked, should be almost an order of magnitude less than the typical offsets. Moreover, if there is both jitter and wander, and you set a very fast poll, you could get low offsets but a high error, because the real error is in the wander. Increasing the loop time constant will report higher offsets, but the time will actually be more correct. This may be complicated in that I'm not sure that the loop time constant is actually clamped by the poll rate, so it is possible that setting a high rate over samples without changing the overall behaviour. Maybe it is clamped by maxpoll? ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Polling interval in FreeBSD vs. Windows
ntp# ntpq -crv -p assID=0 status=01fd leap_none, sync_atomic, 15 events, event_13, version=ntpd 4.2.7p116@1.2439-o Tue Jan 11 23:52:15 UTC 2011 (1), processor=i386, system=FreeBSD/8.1-RELEASE, leap=00, stratum=1, precision=-16, rootdelay=0.000, rootdisp=0.275, refid=PPS, reftime=d0dff52d.2101f848 Tue, Jan 18 2011 11:17:01.128, clock=d0dff52f.32333d40 Tue, Jan 18 2011 11:17:03.196, peer=48630, tc=3, mintc=3, offset=-0.099, frequency=6.980, sys_jitter=0.015, clk_jitter=0.000, clk_wander=0.003 remote refid st t when poll reach delay offset jitter == GPS_NMEA(0) .GPS.0 l4 16 3770.0003.526 18.541 oPPS(0) .PPS.0 l2 16 377 0.000 -0.099 0.015 ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Polling interval in FreeBSD vs. Windows
On Tue, Jan 18, 2011 at 09:02:55AM +, David Woolley wrote: The actual error, when locked, should be almost an order of magnitude less than the typical offsets. Moreover, if there is both jitter and wander, and you set a very fast poll, you could get low offsets but a high error, because the real error is in the wander. Increasing the loop time constant will report higher offsets, but the time will actually be more correct. The trouble is with when locked. When the jitter reaches a certain point (or better the ratio between jitter and clock stability -- usually expressed as Allan intercept in the NTP docs), the PLL won't be able to get a good lock and the clock accuracy will be limited only by the clock stability, not the jitter. As ntpd fixes the time constant to the polling interval, the only thing you can do is to use a lower polling interval. If ntpd was able to change time constant and PLL/FLL mode independently from polling interval, it would be a huge improvement. Would be very tricky though. I'd suggest you to look at the clknetsim graphs, I think you can get a very good understanding how is time and frequency accuracy affected by jitter/wander, poll interval and the PLL gain. http://mlichvar.fedorapeople.org/clknetsim/ -- Miroslav Lichvar ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Polling interval in FreeBSD vs. Windows
Miroslav Lichvar wrote: The trouble is with when locked. When the jitter reaches a certain point (or better the ratio between jitter and clock stability -- usually expressed as Allan intercept in the NTP docs), the PLL won't be able to get a good lock and the clock accuracy will be limited only by the clock stability, not the jitter. Are you suggesting that the assumed Allan intercept if a couple of orders of magnitude too high? As ntpd fixes the time constant to the polling interval, the only thing you can do is to use a lower polling interval. If ntpd was able Linking them makes total sense. The poll interval needs to be a small submultiple of the time constant, so that there is reasonable oversampling and allowance is made for the subsetting of the samples in the initial filter. Polling faster than this adds very little information to the timing solution and polling slower will break the Nyquist criterion. to change time constant and PLL/FLL mode independently from polling interval, it would be a huge improvement. Would be very tricky though. I'd suggest you to look at the clknetsim graphs, I think you can get a very good understanding how is time and frequency accuracy affected by jitter/wander, poll interval and the PLL gain. http://mlichvar.fedorapeople.org/clknetsim/ You are measuring (RMS) offset, which is not the same as error, and you are not accounting for network wander, which can reach 100s of ms, if NTP isn't prioritised. Actually, I suspect the big problem with NTP's theory is that it assumes that both the jitter and wander come from the clock. It breaks down if jitter comes from the measurements (network) and wander from the clock (favours short time constant) or jitter comes from the clock and wander from the network (favours long time constant). ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Polling interval in FreeBSD vs. Windows
On Tue, Jan 18, 2011 at 12:49:52PM +, David Woolley wrote: Miroslav Lichvar wrote: The trouble is with when locked. When the jitter reaches a certain point (or better the ratio between jitter and clock stability -- usually expressed as Allan intercept in the NTP docs), the PLL won't be able to get a good lock and the clock accuracy will be limited only by the clock stability, not the jitter. Are you suggesting that the assumed Allan intercept if a couple of orders of magnitude too high? Yes, when the jitter is too low or the clock too unstable. Ideally, ntp would run a statistic and adjust it in runtime. Chrony counts number of runs of residuals with same sign, perhaps it could work for ntp too. But in ntp the assumed Allan intercept only controls the PLL/FLL switch (and only in daemon mode), the PLL time constant is always fixed to polling interval. As ntpd fixes the time constant to the polling interval, the only thing you can do is to use a lower polling interval. If ntpd was able Linking them makes total sense. The poll interval needs to be a small submultiple of the time constant, so that there is reasonable oversampling and allowance is made for the subsetting of the samples in the initial filter. Polling faster than this adds very little information to the timing solution and polling slower will break the Nyquist criterion. Linking them makes sense if you want to keep things simple and robust. The problem is that even the minimum allowed poll 3 is too long in some situations and that it wastes network bandwidth. to change time constant and PLL/FLL mode independently from polling interval, it would be a huge improvement. Would be very tricky though. I'd suggest you to look at the clknetsim graphs, I think you can get a very good understanding how is time and frequency accuracy affected by jitter/wander, poll interval and the PLL gain. http://mlichvar.fedorapeople.org/clknetsim/ You are measuring (RMS) offset, which is not the same as error, and you are not accounting for network wander, which can reach 100s of ms, if NTP isn't prioritised. Define error and network wander. Please note that the RMS offset is from the actual clock offset maintained by the simulator, not the one reported by ntp. -- Miroslav Lichvar ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Polling interval in FreeBSD vs. Windows
Miroslav Lichvar wrote: Yes, when the jitter is too low or the clock too unstable. Ideally, ntp would run a statistic and adjust it in runtime. Chrony counts It does. I forget the exact metric, but look for the term poll adjust. number of runs of residuals with same sign, perhaps it could work for Linking them makes sense if you want to keep things simple and robust. The problem is that even the minimum allowed poll 3 is too long in some situations and that it wastes network bandwidth. Reducing the poll interval without reducing the time constant will simply result in oversampling. The time constant will still determine the loop behaviour. (It may make things worse if network delay transients no longer fit within the 8 sample filter.) Define error and network wander. It looks like you are actually measuring error, but most people assume that what ntpd reports as offset is the error. Network wander is when, for example, an asymmetric load is applied temporarily. With 3.1 kHz modems, that would result in such a large wander that ntpd would start ignoring the server entirely, so was rather benign. With lower speed DSL, it results in clock steps. With higher speed connections, the peak to peak wander may be under 128ms. Please note that the RMS offset is from the actual clock offset maintained by the simulator, not the one reported by ntp. ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Polling interval in FreeBSD vs. Windows
On Tue, Jan 18, 2011 at 02:11:12PM +, David Woolley wrote: Miroslav Lichvar wrote: Yes, when the jitter is too low or the clock too unstable. Ideally, ntp would run a statistic and adjust it in runtime. Chrony counts It does. I forget the exact metric, but look for the term poll adjust. The allan_xpt variable is 11 by default, and can be changed only by the tinker allan command. (which is what was used to force ntpd to enable FLL in the clknetsim tests) Linking them makes sense if you want to keep things simple and robust. The problem is that even the minimum allowed poll 3 is too long in some situations and that it wastes network bandwidth. Reducing the poll interval without reducing the time constant will simply result in oversampling. The time constant will still determine the loop behaviour. It will improve the accuracy when your Allan intercept is high (e.g. you have a thermally stabilized oscillator). Reducing time constant is probably the more interesting case, but this needs to be done carefully to avoid undersampling, the constant probably could be reduced only to half or quarter, then it would have to switch to FLL. (It may make things worse if network delay transients no longer fit within the 8 sample filter.) Good point. That's another advantage of reducing time constant independently from poll interval. Network wander is when, for example, an asymmetric load is applied temporarily. With 3.1 kHz modems, that would result in such a large wander that ntpd would start ignoring the server entirely, so was rather benign. With lower speed DSL, it results in clock steps. With higher speed connections, the peak to peak wander may be under 128ms. Ok, on the clknetsim page are some tests with temporary asymmetric delays too. It's actually a good example where increasing time constant would allow dropping more than 8 consecutive samples, instead of mangling them with huffpuff to avoid undersampling. -- Miroslav Lichvar ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Polling interval in FreeBSD vs. Windows
Miroslav Lichvar wrote: The allan_xpt variable is 11 by default, and can be changed only by the tinker allan command. (which is what was used to force ntpd to enable FLL in the clknetsim tests) The parameter that is varied with ntpd is the time constant. ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Polling interval in FreeBSD vs. Windows
On 2011-01-18, David Woolley david@ex.djwhome.demon.invalid wrote: unruh wrote: ?? Surely what you want is that your computer's clock shows the correct time. If your computer disciplines the clock to have the true rate (1 sec of computer time equals one second of UTC time) but was 10 hours off, I think most people would count that as a failure of ntp. Not sure what you mean with You cannot do that in an unqualified way. The actual error, when locked, should be almost an order of magnitude less than the typical offsets. Moreover, if there is both jitter and Uh, it depends. If the frequecy shifts ( more or less work being done by the computer) ntp will track very badly. Anyway, if the errors are random, many measurement over a short period will keep the accuracy better than the same number of measurements over a long period. And be more responsive to frequecy shifts. Also ntpd's averaging time is NOT of the order of 100 poll intervals ( which you would need to get your order of magnitude) Especially as ntpd uses only one of every 8 polls, the actual statistical improvement is less than 3, not an order of magnitude. wander, and you set a very fast poll, you could get low offsets but a high error, because the real error is in the wander. Increasing the No. a fast poll also corrects wander faster. ntpd is horribly slow at correcting wander, but doing it quickly corrects it faster. What it does not do is to give a good estimate of wander. The drift correction flops around much more with faster polls, because of the short baseline. That does not matter if you have constant connectivity, but becomes important if you loose connectivity for any time period. loop time constant will report higher offsets, but the time will actually be more correct. sorry, but that is just wrong. The offsets are the best estimate of the correct time Higher offsets means higher errors. This may be complicated in that I'm not sure that the loop time constant is actually clamped by the poll rate, so it is possible that setting a high rate over samples without changing the overall behaviour. Maybe it is clamped by maxpoll? No, it is determined by the number of readings, not by time. More readings in a given time means shorter time constant. Fewer readeings, longer. ntpd throws away 7 out of 8 readings, so the actual poll interval of used reading is 8 times as long as the set poll interval, and the time constant is about 5 times that if I remember correctly. ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Polling interval in FreeBSD vs. Windows
On 2011-01-18, David Woolley david@ex.djwhome.demon.invalid wrote: Miroslav Lichvar wrote: The trouble is with when locked. When the jitter reaches a certain point (or better the ratio between jitter and clock stability -- usually expressed as Allan intercept in the NTP docs), the PLL won't be able to get a good lock and the clock accuracy will be limited only by the clock stability, not the jitter. Are you suggesting that the assumed Allan intercept if a couple of orders of magnitude too high? It depends. There is not one correct Allan intercept. The correct value depends on lots of stuff, including how the computer is used (thermal effects). ntpd assumes one value, measured about 15 years ago. The fact that chrony does 3-20 times better than ntpd at controlling the clock with exactly the same data that ntpd uses suggests that ntpd is far from optimal. As ntpd fixes the time constant to the polling interval, the only thing you can do is to use a lower polling interval. If ntpd was able Linking them makes total sense. The poll interval needs to be a small submultiple of the time constant, so that there is reasonable oversampling and allowance is made for the subsetting of the samples in the initial filter. Polling faster than this adds very little information to the timing solution and polling slower will break the Nyquist criterion. to change time constant and PLL/FLL mode independently from polling interval, it would be a huge improvement. Would be very tricky though. I'd suggest you to look at the clknetsim graphs, I think you can get a very good understanding how is time and frequency accuracy affected by jitter/wander, poll interval and the PLL gain. http://mlichvar.fedorapeople.org/clknetsim/ You are measuring (RMS) offset, which is not the same as error, and you are not accounting for network wander, which can reach 100s of ms, if NTP isn't prioritised. ??? I have never seen 100s of ms. I have rarely seen 1ms. Actually, I suspect the big problem with NTP's theory is that it assumes that both the jitter and wander come from the clock. It breaks down if jitter comes from the measurements (network) and wander from the clock (favours short time constant) or jitter comes from the clock and wander from the network (favours long time constant). ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Polling interval in FreeBSD vs. Windows
On 2011-01-18, David Woolley david@ex.djwhome.demon.invalid wrote: Miroslav Lichvar wrote: Yes, when the jitter is too low or the clock too unstable. Ideally, ntp would run a statistic and adjust it in runtime. Chrony counts It does. I forget the exact metric, but look for the term poll adjust. number of runs of residuals with same sign, perhaps it could work for Linking them makes sense if you want to keep things simple and robust. The problem is that even the minimum allowed poll 3 is too long in some situations and that it wastes network bandwidth. Reducing the poll interval without reducing the time constant will simply result in oversampling. The time constant will still determine the loop behaviour. (It may make things worse if network delay transients no longer fit within the 8 sample filter.) If the error is random error, then oversampling is exactly what you want. It knocks down the error by the square root of the oversampling. Define error and network wander. It looks like you are actually measuring error, but most people assume that what ntpd reports as offset is the error. Network wander is when, for example, an asymmetric load is applied temporarily. With 3.1 kHz modems, that would result in such a large wander that ntpd would start ignoring the server entirely, so was rather benign. With lower speed DSL, it results in clock steps. With higher speed connections, the peak to peak wander may be under 128ms. In none of my tests-- both ethernet in the university, or adsl/cable modem from home ( over commercial telephone/cable tv lines with their huge variability of load) have I ever seen such behaviour. Please note that the RMS offset is from the actual clock offset maintained by the simulator, not the one reported by ntp. ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Polling interval in FreeBSD vs. Windows
unruh wrote: are not accounting for network wander, which can reach 100s of ms, if NTP isn't prioritised. ??? I have never seen 100s of ms. I have rarely seen 1ms. A fudge option was added because of this effect. As I detailed elsewhere on the thread, originally the wander tended to be so large that ntpd would reject the source, but as networks speeded up, the result was 500 ms positive and negative steps. It is probably under the 128 ms for most people these days. I certainly observed it. ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Polling interval in FreeBSD vs. Windows
unruh wrote: more responsive to frequecy shifts. Also ntpd's averaging time is NOT of the order of 100 poll intervals ( which you would need to get your order Fair comment. However, it is still measuring the difference between a filtered estimate of true time and an instantaneous measurement. It is not measuring the difference between true time and the system clock time, which is how a lot of people seem to treat it. of magnitude) Especially as ntpd uses only one of every 8 polls, the actual statistical improvement is less than 3, not an order of magnitude. No. a fast poll also corrects wander faster. ntpd is horribly slow at correcting wander, but doing it quickly corrects it faster. What it does Depends whether the wander is in the clock or the measurement. loop time constant will report higher offsets, but the time will actually be more correct. sorry, but that is just wrong. The offsets are the best estimate of the correct time Higher offsets means higher errors. The offsets are the difference between the best estimate of the correct time, before it is updated, and the time from a specific, selected measurement. This may be complicated in that I'm not sure that the loop time constant is actually clamped by the poll rate, so it is possible that setting a high rate over samples without changing the overall behaviour. Maybe it is clamped by maxpoll? No, it is determined by the number of readings, not by time. More readings in a given time means shorter time constant. Fewer readeings, I am pretty certain that they are decoupled internally, and the degree of over-sampling can vary. Essentially, though, it is the time constant that is tuned, and the poll interval has to over sample that and the 1 out of 8 filter. longer. ntpd throws away 7 out of 8 readings, so the actual poll interval of used reading is 8 times as long as the set poll interval, and the time constant is about 5 times that if I remember correctly. ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Polling interval in FreeBSD vs. Windows
On Tue, Jan 18, 2011 at 07:18:08PM +, unruh wrote: On 2011-01-18, David Woolley david@ex.djwhome.demon.invalid wrote: The actual error, when locked, should be almost an order of magnitude less than the typical offsets. Moreover, if there is both jitter and Uh, it depends. If the frequecy shifts ( more or less work being done by the computer) ntp will track very badly. Anyway, if the errors are random, many measurement over a short period will keep the accuracy better than the same number of measurements over a long period. And be more responsive to frequecy shifts. Also ntpd's averaging time is NOT of the order of 100 poll intervals ( which you would need to get your order of magnitude) Especially as ntpd uses only one of every 8 polls, the actual statistical improvement is less than 3, not an order of magnitude. It seems to depend on the distribution of network delays too. With normally distributed delays I see improvement about 3, but with exponentially distributed delays it's slightly more than 10. Is that because the selected sample is the one with the best delay, so it carries more information than the others? -- Miroslav Lichvar ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Polling interval in FreeBSD vs. Windows
On Tue, Jan 18, 2011 at 19:18 PM, unruh un...@wormhole.physics.ubc.ca wrote: On 2011-01-18, David Woolley david@ex.djwhome.demon.invalid wrote: wander, and you set a very fast poll, you could get low offsets but a high error, because the real error is in the wander. Increasing the No. a fast poll also corrects wander faster. ntpd is horribly slow at correcting wander, but doing it quickly corrects it faster. I would agree if you said perceived wander with measurement-induced error. In other words, while turning the slew knob faster reduces your offsets faster, doing so risks following measurement noise rather than the underlying trend. What it does not do is to give a good estimate of wander. The drift correction flops around much more with faster polls, because of the short baseline. That does not matter if you have constant connectivity, but becomes important if you loose connectivity for any time period. It matters to me. I want my frequency slewing to ignore measurement noise, not react quickly to it. This may be complicated in that I'm not sure that the loop time constant is actually clamped by the poll rate, so it is possible that setting a high rate over samples without changing the overall behaviour. Maybe it is clamped by maxpoll? No, it is determined by the number of readings, not by time. More readings in a given time means shorter time constant. Fewer readeings, longer. ntpd throws away 7 out of 8 readings, so the actual poll interval of used reading is 8 times as long as the set poll interval, and the time constant is about 5 times that if I remember correctly. You do not understand NTP's clock filter. As each new sample is pushed in, only one of the prior 8 samples is used for the moment, but each sample has 8 chances to be that favorite. That is not equivalent to throwing away 7 out of 8 readings on my planet. Cheers, Dave Hart ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Polling interval in FreeBSD vs. Windows
unruh wrote: If the error is random error, then oversampling is exactly what you want. It knocks down the error by the square root of the oversampling. But the noise reduction is in the high frequencies, which are suppressed by the, time constant based, loop filter, so the improvement is at jitter frequencies that are already significantly suppressed. In none of my tests-- both ethernet in the university, or adsl/cable modem from home ( over commercial telephone/cable tv lines with their huge variability of load) have I ever seen such behaviour. I certainly have. I'm not sure if the DSL was 0.5 or 1Mb/s at the time. I actually patched the maximum root distance to be just larger than that on a quiet link, but the official fix was to add tinker huffpuff (I wrongly wrote fudge earlier in this thread). ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Polling interval in FreeBSD vs. Windows
Dave Hart wrote: I would agree if you said perceived wander with measurement-induced error. In other words, while turning the slew knob faster reduces your offsets faster, doing so risks following measurement noise rather than the underlying trend. That was my point. Unruh's main issue is that, on modern LANs, the dominant low frequency error is in the local clock, rather than the measurements. You do not understand NTP's clock filter. As each new sample is pushed in, only one of the prior 8 samples is used for the moment, but each sample has 8 chances to be that favorite. That is not equivalent to throwing away 7 out of 8 readings on my planet. It's more complicated. I don't think the current version ages the samples, so, for a concrete example, if you have a downwards sawtooth, in delay, of period 16, you will get all the first 16, the last of these will be repeated 8 times. When it drops out of the filter, one will be on the 8th entry in the sawtooth, so you will get another run of 9 different samples, so it will throw away 7 in 16 on the long term average. I think, if you increase the sawtooth period, it will be 7 in n. Obviously, these are contrived cases. I think the filter will take out more like 7 in 8 for gaussian input, but the expected input pattern isn't actually gaussian, either. ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Polling interval in FreeBSD vs. Windows
At 02:18 PM 1/18/2011, unruh wrote... Especially as ntpd uses only one of every 8 polls, Is this relatively recent behavior? I can't provide NTP version #s immediately, but it used to be I could watch ntpq -p, and see the offset and jitter change with each poll. More recently, I don't see this. I haven't counted, but 1 of 8 sounds about right. Is it ignoring 7 of 8 (if so, why, instead of 8x polling interval?), or is it averaging the 8, then using the average? ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Polling interval in FreeBSD vs. Windows
On Wed, Jan 19, 2011 at 02:47 UTC, Mike S mi...@flatsurface.com wrote: At 02:18 PM 1/18/2011, unruh wrote... Especially as ntpd uses only one of every 8 polls, Is this relatively recent behavior? No. See the Clock filter algorithm slide 13 in: http://www.eecis.udel.edu/~mills/database/brief/arch/arch.pdf I can't provide NTP version #s immediately, but it used to be I could watch ntpq -p, and see the offset and jitter change with each poll. More recently, I don't see this. I haven't counted, but 1 of 8 sounds about right. Is it ignoring 7 of 8 (if so, why, instead of 8x polling interval?), or is it averaging the 8, then using the average? Use: ntpq -p -c rv 1 to see the detailed peer variables for the first association, or change the 1 to 2 for the second association, etc. Among those are the 8 clock filter entries (or 24 since each entry has a dispersion, delay, and apparent offset). Whichever entry has the lowest delay is presumed (wisely IMO) to have the least error in the apparent offset. Cheers, Dave Hart ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Polling interval in FreeBSD vs. Windows
On 2011-01-19, Mike S mi...@flatsurface.com wrote: At 02:18 PM 1/18/2011, unruh wrote... Especially as ntpd uses only one of every 8 polls, Is this relatively recent behavior? I can't provide NTP version #s Nope. It is very old behaviour. It is the filter at the beginning whose purpose is to select from the last 8 that item whose round trip time is smallest in the hope that this has the least assymetry in the outgoing and incoming times. If there is sever assymmetry then this may help. In many cases it only succeeds in throwing away 88 percent of the data collected. immediately, but it used to be I could watch ntpq -p, and see the offset and jitter change with each poll. More recently, I don't see this. I haven't counted, but 1 of 8 sounds about right. Is it ignoring 7 of 8 (if so, why, instead of 8x polling interval?), or is it averaging the 8, then using the average? It takes the one with the least round trip time of the last 8. Thus exactly which one is chosen is somewhat random. ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Polling interval in FreeBSD vs. Windows
Mike S wrote: Is this relatively recent behavior? I can't provide NTP version #s immediately, but it used to be I could watch ntpq -p, and see the offset and jitter change with each poll. More recently, I don't see this. I haven't counted, but 1 of 8 sounds about right. I believe the difference is that ntpd v3 used to add a dispersion figure to the delay, thus favouring newer samples, even though they had higher delays. ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Polling interval in FreeBSD vs. Windows
On Sun, Jan 16, 2011 at 05:37:18PM -0800, Chris Albertson wrote: A longer poling interval is not a bad thing. The polling interval is adjusted so as to reduce total noise. There is a sweet spot where polling faster or slower is worse. Yes, there is a sweet spot, but ntpd isn't looking for it. It strongly prefers longer polling interval to save network bandwidth. If you want the best accuracy, you will need to set maxpoll according to the network jitter and clock stability you have. For a typical clock oscillator and the standard kernel PLL, poll 3 will give you better accuracy than poll 4 when the network jitter is about 100 microseconds or less. Such jitter is not uncommon on LAN, sometimes I observe 100us jitter to close pool.ntp.org servers! As an example, lets say you wanted to measure the thickness of a sheet of paper but your ruller only goes to 1/100 inch divisions. You get soe gross errors if yu tried to measure one sheet. But stack 1,000 sheets and you will do well. Longer polling interval works kind of the same way. Except the thickness is slowly changing during the production, so you have to use a compromise to keep the noise down and to get the current thinkness. I think the longer poll time is telling you something good about the internal clock in the BSD system. What exactly? -- Miroslav Lichvar ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Polling interval in FreeBSD vs. Windows
On 2011-01-17, Miroslav Lichvar mlich...@redhat.com wrote: On Sun, Jan 16, 2011 at 05:37:18PM -0800, Chris Albertson wrote: A longer poling interval is not a bad thing. The polling interval is adjusted so as to reduce total noise. There is a sweet spot where polling faster or slower is worse. This is a bit of a myth. It depends entirely on what you want to optimize. To optimize the offset error, more polling is better. If you want to optimize the drift rate, then there is a sweet spot but it depends on all sorts of stuff, including the daily temp fluctuations due to the computer's being used more during the say. It is not at all clear what ntp is set up to optimize but as stated below, reducing network use is part of it. It also chooses an Allan intercept which is not one measured for your system but some average value ( from 1995?). Yes, there is a sweet spot, but ntpd isn't looking for it. It strongly prefers longer polling interval to save network bandwidth. If you want the best accuracy, you will need to set maxpoll according to the network jitter and clock stability you have. For a typical clock oscillator and the standard kernel PLL, poll 3 ??? Poll 3? will give you better accuracy than poll 4 when the network jitter is about 100 microseconds or less. Such jitter is not uncommon on LAN, sometimes I observe 100us jitter to close pool.ntp.org servers! As an example, lets say you wanted to measure the thickness of a sheet of paper but your ruller only goes to 1/100 inch divisions. You get soe gross errors if yu tried to measure one sheet. But stack 1,000 sheets and you will do well. Longer polling interval works kind of the same way. Except the thickness is slowly changing during the production, so you have to use a compromise to keep the noise down and to get the current thinkness. I think the longer poll time is telling you something good about the internal clock in the BSD system. What exactly? ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Polling interval in FreeBSD vs. Windows
Miroslav Lichvar wrote: On Sun, Jan 16, 2011 at 05:37:18PM -0800, Chris Albertson wrote: Yes, there is a sweet spot, but ntpd isn't looking for it. It strongly prefers longer polling interval to save network bandwidth. If you want I believe it is intended to select the sweet spot, although it is possible that it has been left behind by technology changes. the best accuracy, you will need to set maxpoll according to the network jitter and clock stability you have. For a typical clock oscillator and the standard kernel PLL, poll 3 will give you better accuracy than poll 4 when the network jitter is about 100 microseconds or less. Such jitter is not uncommon on LAN, sometimes I observe 100us jitter to close pool.ntp.org servers! I suspect you are using offset as your figure of merit. You cannot do that in an unqualified way. ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Polling interval in FreeBSD vs. Windows
On 2011-01-17, David Woolley david@ex.djwhome.demon.invalid wrote: Miroslav Lichvar wrote: On Sun, Jan 16, 2011 at 05:37:18PM -0800, Chris Albertson wrote: Yes, there is a sweet spot, but ntpd isn't looking for it. It strongly prefers longer polling interval to save network bandwidth. If you want I believe it is intended to select the sweet spot, although it is possible that it has been left behind by technology changes. the best accuracy, you will need to set maxpoll according to the network jitter and clock stability you have. For a typical clock oscillator and the standard kernel PLL, poll 3 will give you better accuracy than poll 4 when the network jitter is about 100 microseconds or less. Such jitter is not uncommon on LAN, sometimes I observe 100us jitter to close pool.ntp.org servers! I suspect you are using offset as your figure of merit. You cannot do that in an unqualified way. ?? Surely what you want is that your computer's clock shows the correct time. If your computer disciplines the clock to have the true rate (1 sec of computer time equals one second of UTC time) but was 10 hours off, I think most people would count that as a failure of ntp. Not sure what you mean with You cannot do that in an unqualified way. ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Polling interval in FreeBSD vs. Windows
On Sun, Jan 16, 2011 at 22:34 UTC, Edward T. Mischanko etm1...@hotmail.com wrote: Why in ...4.2.7p116 is it that I can set minpoll to 3 and poll every 8 seconds in Windows, but when I try the same thing for my referrence clock in FreeBSD, it will poll no sooner than 16 seconds. On Mon, Jan 17, 2011 at 00:47 UTC, Edward T. Mischanko etm1...@hotmail.com wrote: I am running the same version of ntpd on both boxes. I can not reproduce your results: hart@psp-fb2 fgrep 'minpoll 3' /etc/ntp.conf server 127.127.8.0 minpoll 3 mode 7 noselect hart@psp-fb2 ntpq -crv -p associd=0 status=0018 leap_none, sync_unspec, 1 event, no_sys_peer, version=ntpd 4.2.7p118@1.2449-o Tue Jan 18 01:07:14 UTC 2011 (1), processor=i386, system=FreeBSD/6.4-STABLE, leap=00, stratum=3, precision=-19, rootdelay=1.019, rootdisp=950.713, refid=149.20.68.27, reftime=d0df67eb.04b05ebb Tue, Jan 18 2011 1:14:19.018, clock=d0df6822.45ee9a50 Tue, Jan 18 2011 1:15:14.273, peer=0, tc=6, mintc=3, offset=0.726, frequency=80.705, sys_jitter=0.576, clk_jitter=0.257 remote refid st t when poll reach delay offset jitter === GENERIC(0) .GPS.0 l58 3770.000 -22.304 4.303 [...] Perhaps someone else can give it a try, or you could verify your results with minpoll 3 on FreeBSD. Cheers, Dave Hart ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
[ntp:questions] Polling interval in FreeBSD vs. Windows
Why in ...4.2.7p116 is it that I can set minpoll to 3 and poll every 8 seconds in Windows, but when I try the same thing for my referrence clock in FreeBSD, it will poll no sooner than 16 seconds. ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Polling interval in FreeBSD vs. Windows
Edward T. Mischanko wrote: Why in ...4.2.7p116 is it that I can set minpoll to 3 and poll every 8 seconds in Windows, but when I try the same thing for my referrence clock in FreeBSD, it will poll no sooner than 16 seconds. Because reference clocks override the poll interval limits. Why do you feel the need to go outside the recommended limits? ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Polling interval in FreeBSD vs. Windows
That is the problem exactly! In FreeBSD I can not set anything to minpoll 3, but in Windows I can. Why? If I could poll my GPS/PPS every 8 seconds, I most likely would. ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Polling interval in FreeBSD vs. Windows
On Sun, Jan 16, 2011 at 22:34 UTC, Edward T. Mischanko etm1...@hotmail.com wrote: Why in ...4.2.7p116 is it that I can set minpoll to 3 and poll every 8 seconds in Windows, but when I try the same thing for my referrence clock in FreeBSD, it will poll no sooner than 16 seconds. Presumably because this was changed at some point after the version of ntpd that your FreeBSD box uses. Cheers, Dave Hart ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Polling interval in FreeBSD vs. Windows
I am running the same version of ntpd on both boxes. ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Polling interval in FreeBSD vs. Windows
On 1/16/2011 6:02 PM, Edward T. Mischanko wrote: That is the problem exactly! In FreeBSD I can not set anything to minpoll 3, but in Windows I can. Why? If I could poll my GPS/PPS every 8 seconds, I most likely would. Why? Polling that fast would almost certainly diminish NPTD's ability to give you the maximum possible accuracy. I over simplify, perhaps grossly, but short poll intervals set your clock fairly quickly while longer poll intervals set your clock more accurately. NTPD will do the right thing if you allow it to do its job! ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
Re: [ntp:questions] Polling interval in FreeBSD vs. Windows
A longer poling interval is not a bad thing. The polling interval is adjusted so as to reduce total noise. There is a sweet spot where polling faster or slower is worse. As an example, lets say you wanted to measure the thickness of a sheet of paper but your ruller only goes to 1/100 inch divisions. You get soe gross errors if yu tried to measure one sheet. But stack 1,000 sheets and you will do well. Longer polling interval works kind of the same way. I think the longer poll time is telling you something good about the internal clock in the BSD system. -- = Chris Albertson Redondo Beach, California ___ questions mailing list questions@lists.ntp.org http://lists.ntp.org/listinfo/questions
