Re: [time-nuts] looking for good description/generalized model for time adjustments
This has been a fun problem to think about. Thanks. Let's see if I under stand things... You have code running on a system with a crappy clock. This system is the Master. It's clock is, by definition, the reference that runs everything. The crappy clock has temperature fluctuations on the ballpark of an hour. Temperature and hence frequency is roughly a square wave so time offset (relative to UTC) will be triangular. Master talks to devices with clocks that are much better than Master's clock. Master can tell the devices to "Do X at time T". (Where T is Master time.) One of the Xs is to wiggle a wire that the master can get a good timestamp on. The master can use that to read the device clocks. Master wants to say to Device A, "Do A at time T1" where T1 is Master time. So it actually says "Do A at time T1+a" where a is the fudge factor to correct for the clock offsets. Master needs to know a accurately enough so that A happens within a ms of T1. Suppose the offset of Master from UTC is M(t) and the offset of Device A is A(t). From Master's point of view, the clock on A will look like it has an offset of A(t)-M(t). (If Master's clock is fast, it looks like A's clock is slow.) I'm assuming you (somehow) have a rough idea of a. Plan 1: Shortly before T1, ask A what time it is to get an updated reading on a. Then tell A, "Do A at time T1+a" with the up-to-date a. I think you will have to poll A regularly to track the clock. I'm thinking of, say, once per second. As long as you can poll fast enough this should work. You can work out the math on how fast you have to poll. You need the max error on reading the clock and the max drift time between reading a and using it. "Fast enough" isn't really the right term. You could poll slower to roughly track a, but then slip in an extra calibration cycle just before you need an accurate a to send the command. I think this is just a simple PLL with a fast time constant (high cutoff frequency) on the filter. Plan 2: If that's not good enough, you have to model the clock. Hopefully that's as simple as estimating the frequency offset as well as estimating the time offset. That frequency offset is a square wave with rounded corners. I'm not sure what happens if the corners are too sharp. I'm assuming the corners are rounded enough that you can track them. The idea would be to say "Do A at time T+a0+a1" where a0 is the a above and a1 is current freq-offset multiplied by the time between measurement and T. I'm not sure how to think about this as a PLL. The filter cutoff frequency would be lower than above, averaging over several samples. But it has to be fast enough to track the corners. -- These are my opinions, not necessarily my employer's. I hate spam. ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] looking for good description/generalized model for time adjustments
-Original Message- From: time-nuts-boun...@febo.com [mailto:time-nuts-boun...@febo.com] On Behalf Of Hal Murray Sent: Wednesday, July 29, 2009 2:51 PM To: Discussion of precise time and frequency measurement Subject: Re: [time-nuts] looking for good description/generalized model for time adjustments > >> Precisely so. And NTP may actually be the best model here. Does > NTP's "corrected output" meet the "must be monotonic and not > discontinuous" criteria (being too lazy to just go read the NTP docs, > which I have, and which I'll take a look at after lunch). There is a lot of info available about ntp, you may have troubles finding what you want, especially if you don't already know what you are looking for. I'm being sloppy when I say "ntp". It's both a protocol spec (RFC-whatever) and a reference implementation (ntpd) that is widely deployed. >>>> What OS are you using? FreeBSD is very good. Linux is not--so-good. >>> RTEMS and/or VxWorks (I'm using RTEMS, others with whom we communicate use >>> a variety of other things. And we don't have network connections per se.. >>> That's why I'm looking for more generic descriptions that aren't tied to >>> things like packet definitions or peculiarities of IP routing. Basically, >>> all of these things should be able to be boiled down to two things: a >>> synchronization signal and a synchronization message. The other mode is using a refclock. ntpd includes support of over 20 different types of clocks, many are no longer interesting. The key to getting good time is something like a PPS pulse and kernel support to grab a timestamp in the interrupt routing. There is also a batch of PLL code in the kernel that I don't understand. >>>> and it is that PLL code that is particularly interesting (Poul-Henning has >>>> useful information in kern/time_tc.c, etc.) For network traffic, ntp assumes the delays are symmetric. >>> In my application, we don't need to "probe the network" to figure out the >>> latencies. We know them apriori, and they are also "very small" compared to >>> the required time precision. The part I'm interested in is the automated >>> reconciliation of the always varying clocks/oscillators on the platforms, >>> essentially trying to predict a bad clock with a good one. > ---> But it *is* the master, by fiat. Here's a scenario: A schedule > is published that says: (MT = Master's time) > At 12:00.001MT Box A puts out a pulse > At 12:00.002MT Box B puts out a pulse > At 12:01.001MT Box A puts out a pulse. > Box A and Box B MUST follow Master Time, no matter how crummy it is. ntp is trying for UTC, the one great time. But that comes from outside ntp. You can simplify things a lot if you tell it (config file) to only use one server. For example, you could setup box A as the master and tell B to sync to it. It may be better to setup another box in a stable environment and sync both A and B to it. That gives B a stable target. --> Can't do that in this environment. The master is the master, and all the slaves are fore-ordained and must follow it as best they can. What I need to do, really, is figure out what "best they can" is. > A Single board computer with non TCXO clock (on order of 10ppm > variability over short term) is the "system controller" and sets the > time schedules. It sends commands to devices which have TCXO clocks > (on order of 1ppm short term variability) saying "at time X do action > Y". It also periodically sends messages to the devices saying "At the > tone, my time is Z". How good is your netework connection? >>>> Negligible latency (at least in the context of the required time >>>> precision). In reality, it has a worst case jitter of about 1 microsecond >>>> and a mean latency of 0.5 microsecond (that is, a "tick" on one end of the >>>> link appears at the other end of the link with a uniformly distributed >>>> delay of 0-1 microsecond) >>>> Think of it as if I had a piece of wire connecting the boxes, to do with >>>> whatever I want. And then, on the side, I have a way to transmit messages >>>> among boxes (which has greater latency and jitter) 10 ppm short term variability seems huge. The crystals I've looked at (in PCs) are ballpark of 1 ppm per C. Do you have wide temperature changes short term or nasty crystals? (or am I lucky?) >>> wide temperature ranges: The environment where you go from full sunlight >>> (at 1.3kW/square meter) to full
Re: [time-nuts] looking for good description/generalized model for time adjustments
> >> Precisely so. And NTP may actually be the best model here. Does > NTP's "corrected output" meet the "must be monotonic and not > discontinuous" criteria (being too lazy to just go read the NTP docs, > which I have, and which I'll take a look at after lunch). There is a lot of info available about ntp, you may have troubles finding what you want, especially if you don't already know what you are looking for. I'm being sloppy when I say "ntp". It's both a protocol spec (RFC-whatever) and a reference implementation (ntpd) that is widely deployed. ntp tries to keep the local machine in sync with UTC. The model is that the world is a forest of trees. Each tree starts with a local source of time that claims to be UTC. ntp calls them refclocks. Each tree is a collection of servers. The refclocks are stratum 0. The root of each tree is stratum 1. The next layer of servers is stratum 2... A lot of the code in ntp is figuring out which trees to believe and then selecting the best one to sync to. ntpd is both a client and server. It's a server distributing time to clients at a higher stratum and a client getting time from servers at a lower stratum. What OS are you using? FreeBSD is very good. Linux is not--so-good. ntp runs in two modes. Normally, it gets the time over the net. Ballpark accuracies are 1 ms if you have a good local system to talk to and a lightly loaded network and 10s to 100 ms if you go out over the big bad internet. The other mode is using a refclock. ntpd includes support of over 20 different types of clocks, many are no longer interesting. The key to getting good time is something like a PPS pulse and kernel support to grab a timestamp in the interrupt routing. There is also a batch of PLL code in the kernel that I don't understand. For network traffic, ntp assumes the delays are symmetric. You can confuse it with something like a big download over a DSL link. Delays are not a problem. It's jitter and asymmetry. After a packet exchange, you end up with 4 time stamps. When the request left the client. When it arrived at the server. When the response left the server. When it arrived back at the client. (The packet can sit on the server for a while, say if it is busy doing other things.) If you subtract off the time the packet sat on the server you have the round trip time. If you assume symmetry you can get the offset. > ---> But it *is* the master, by fiat. Here's a scenario: A schedule > is published that says: (MT = Master's time) > At 12:00.001MT Box A puts out a pulse > At 12:00.002MT Box B puts out a pulse > At 12:01.001MT Box A puts out a pulse. > Box A and Box B MUST follow Master Time, no matter how crummy it is. ntp is trying for UTC, the one great time. But that comes from outside ntp. You can simplify things a lot if you tell it (config file) to only use one server. For example, you could setup box A as the master and tell B to sync to it. It may be better to setup another box in a stable environment and sync both A and B to it. That gives B a stable target. > A Single board computer with non TCXO clock (on order of 10ppm > variability over short term) is the "system controller" and sets the > time schedules. It sends commands to devices which have TCXO clocks > (on order of 1ppm short term variability) saying "at time X do action > Y". It also periodically sends messages to the devices saying "At the > tone, my time is Z". How good is your netework connection? You can get better data by sending several packets and picking the best one. In this context that probably means the one with the lowest round-trip-time. Lowest time means lowest queuing delays. That assumes no-queuing would be symmetric. ntp is trying to minimize network load. You may do better by sending more packets. (there is a maxpoll and a burst mode in the config file) If I was working on something like that, I would setup a testbed and collect some data. 10 ppm short term variability seems huge. The crystals I've looked at (in PCs) are ballpark of 1 ppm per C. Do you have wide temperature changes short term or nasty crystals? (or am I lucky?) It sounds a little backwards to have the machine with the lousy clock be the one in charge. Can it get the time from any of the devices with better clocks? > What we would like is a) some algorithms to do this (and NTP type PLLs > are a decent way) and b) some formalism and rigor to choose operating > parameters (e.g. update every 1 second or every day or whatever) There is a lot of formalism with ntp. I'm not familiar with most of it. Picking parameters is a black art. The ntp problem is the same as a GPSDO: you get good long term stability. Short to medium is where things get interesting. -- These are my opinions, not necessarily my employer's. I hate spam. ___ time-nuts mailing list -- time-nuts@febo.com To
Re: [time-nuts] looking for good description/generalized model for time adjustments
-Original Message-
From: time-nuts-boun...@febo.com [mailto:time-nuts-boun...@febo.com] On Behalf
Of Christian Vogel
Sent: Wednesday, July 29, 2009 11:51 AM
To: Discussion of precise time and frequency measurement
Subject: Re: [time-nuts] looking for good description/generalized model for
time adjustments
Hi Lux,
> The real problem is that the "master" clock might be a crummy 50ppm XO, and
> all us "slaves" all have TCXOs that are good to a few ppm. It's almost the
> inverse problem from disciplining a XO with GPS.
>
I have the impression that you are confused by your own weird
terminology ;-). If you have a rather good clock yourself, and want to
somehow characterize external events with their own sketchy time-of-day
estimation, then please don't call this cheap thing the "master".
---> But it *is* the master, by fiat. Here's a scenario: A schedule is
published that says:
(MT = Master's time)
At 12:00.001MT Box A puts out a pulse
At 12:00.002MT Box B puts out a pulse
At 12:01.001MT Box A puts out a pulse.
Box A and Box B MUST follow Master Time, no matter how crummy it is.
Just let your internal clock ("precise") run undisturbed and capture the
precise time for all the external events received from the "cheap"
outside machines. Then timestamp a number of external/internal events
and calculate the average relative external clock frequency. Filter the
last N events and use the relative frequencies to extrapolate events to
be expected in the future. It's basically just a PLL done in software.
--- That is the general solution. What I'm looking for is a bit more formalism
(presumably someone has studied this and done a paper?) And, (so I don't have
to write them) if someone has done some nice figures and examples..
--specific example..
A Single board computer with non TCXO clock (on order of 10ppm variability over
short term) is the "system controller" and sets the time schedules. It sends
commands to devices which have TCXO clocks (on order of 1ppm short term
variability) saying "at time X do action Y". It also periodically sends
messages to the devices saying "At the tone, my time is Z".
What we would like is a) some algorithms to do this (and NTP type PLLs are a
decent way) and b) some formalism and rigor to choose operating parameters
(e.g. update every 1 second or every day or whatever)
___
time-nuts mailing list -- time-nuts@febo.com
To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts
and follow the instructions there.
Re: [time-nuts] looking for good description/generalized model for time adjustments
-Original Message- From: time-nuts-boun...@febo.com [mailto:time-nuts-boun...@febo.com] On Behalf Of Hal Murray Sent: Wednesday, July 29, 2009 11:45 AM To: Discussion of precise time and frequency measurement Subject: Re: [time-nuts] looking for good description/generalized model for time adjustments > I'm looking for a good (short) description and/or a generalized model > for relating a local time counter to some external reference. Here's > the scenario.. A computer has a local clock that is a counter being > driven by a local oscillator. Periodically, we get "time updates" from > some outside source that give an "absolute time" and a sync. What sort of accuracy do you want? seconds? milliseconds? microseconds? >>>> milliseconds over days (i.e. parts per billion).. but that's not an >>>> absolute accuracy requirement.. we need to have multiple boxes >>>> "synchronized" to a common time scale so that they can generate an event >>>> within a millisecond, over a time span of, say, 20 minutes. (which is on >>>> the order of 1 ppm). What sort of computer? A big one where you can run lots of software or a tiny one where every instruction counts? How stable is the environment (temperature)? >>>> I'm really looking for a generalized model, not tied to a specific >>>> implementation. BUT, in this case, we can run a fair amount of software. >>>> The environment is not stable (hence the variation of frequency of the >>>> sources is substantial) The trick is to think of frequency rather than time. The usual problem with clock drift is that the local oscillator is off slightly, typically a few 10s of ppm. You can fix most of that if you can fudge the frequency of the clock. Say your system is updating the time from an interrupt that goes off every ms and you are keeping time in ms. >>> yes, that's basically calibrating the "clock rate" If you want to get the human out of the loop, you can implement a PLL. That gets complicated with filtering and stability and such. >> Precisely so. And NTP may actually be the best model here. Does NTP's >> "corrected output" meet the "must be monotonic and not discontinuous" >> criteria (being too lazy to just go read the NTP docs, which I have, and >> which I'll take a look at after lunch). ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] looking for good description/generalized model for time adjustments
Hi Lux,
The real problem is that the "master" clock might be a crummy 50ppm XO, and all us
"slaves" all have TCXOs that are good to a few ppm. It's almost the inverse problem from
disciplining a XO with GPS.
I have the impression that you are confused by your own weird
terminology ;-). If you have a rather good clock yourself, and want to
somehow characterize external events with their own sketchy time-of-day
estimation, then please don't call this cheap thing the "master".
Just let your internal clock ("precise") run undisturbed and capture the
precise time for all the external events received from the "cheap"
outside machines. Then timestamp a number of external/internal events
and calculate the average relative external clock frequency. Filter the
last N events and use the relative frequencies to extrapolate events to
be expected in the future. It's basically just a PLL done in software.
Without telling us some details, noone here will be able to give you
real advice, I think...
Chris
___
time-nuts mailing list -- time-nuts@febo.com
To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts
and follow the instructions there.
Re: [time-nuts] looking for good description/generalized model for time adjustments
> I'm looking for a good (short) description and/or a generalized model
> for relating a local time counter to some external reference. Here's
> the scenario.. A computer has a local clock that is a counter being
> driven by a local oscillator. Periodically, we get "time updates" from
> some outside source that give an "absolute time" and a sync.
What sort of accuracy do you want? seconds? milliseconds? microseconds?
What sort of computer? A big one where you can run lots of software or a
tiny one where every instruction counts? How stable is the environment
(temperature)?
The trick is to think of frequency rather than time.
The usual problem with clock drift is that the local oscillator is off
slightly, typically a few 10s of ppm. You can fix most of that if you can
fudge the frequency of the clock. Say your system is updating the time from
an interrupt that goes off every ms and you are keeping time in ms.
The old code would look something like:
ms = ms + 1
if ms == 1000 { sec = sec + 1; ms = 0}
The first step is to change that to:
ns = ns + 100
if ns == 100 {ms = ms + 1; ns = 0}
if ms == 1000 { sec = sec + 1; ms = 0}
The trick is you want to change that first 100 to match what the osc is
actually delivering. So the code looks like:
initialize: nspertick = 100 + ppmfudge
run time:
ns = ns + nspertick
while ns >= 100 {ms = ms + 1; ns = ns - 100}
if ms == 1000 { sec = sec + 1; ms = 0}
if ppmfudge is negative, you will skip occasional updates to ms (drop ticks).
if ppmfudge is positive, you will occasionally jump by 2 ms.
I picked ms and ns for this example since the numbers work out conveniently
for an integer fudge factor in ppm. You will need more bits for finer
control.
1 ppm is 0.08 seconds per day or 1/2 second per week.
With that sort of setup, and a stable clock (temperature?) you can get within
a few seconds per week by hand.
If you want to get the human out of the loop, you can implement a PLL. That
gets complicated with filtering and stability and such.
Have you looked into ntp? The fudge factor is usually called drift. It's
usually stored in /etc/ntp/drift or /var/lib/ntp/drift or something like
that. With a good local clock (gps, pps), you can easily track temperature.
(I'll say more if anybody wants.)
Historical trivia:
I first learned about this back in 1976 or early 1977. Shortly after I got
to Xerox, Ed Taft fixed a bug in the Alto OS by adding that code. The
machine was designed with a 170 ns cycle time. Everybody knew that,
including the guys who wrote the timekeeping code.
You can't buy 170ns crystals. They come in MHz. So they got 5.88 MHz. If I
did the math right, that's off by 340 pm or 29 seconds per day. (That feels
about right. 30 seconds per day is horrible for a mature system, but close
enough that many other things will be more important during the early
days/years of a project.)
More trivia: The early 3 megabit Ethernet was actually 5.88 / 2.
--
These are my opinions, not necessarily my employer's. I hate spam.
___
time-nuts mailing list -- time-nuts@febo.com
To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts
and follow the instructions there.
Re: [time-nuts] looking for good description/generalized model for time adjustments
-Original Message- From: time-nuts-boun...@febo.com [mailto:time-nuts-boun...@febo.com] On Behalf Of J. Forster Sent: Wednesday, July 29, 2009 10:19 AM To: Discussion of precise time and frequency measurement Subject: Re: [time-nuts] looking for good description/generalized model for time adjustments Maybe let the clocks run unadjusted all day and resync between fixed times once a day? This is what the power companies do. You can back-figure the precise times if you really need to from the once a day correction files. This assumes the clock rates are just off a bit, and do not randomly fluctuate. -John --- If it was just a matter of "post processing" the data to figure out from recorded times when things actually happened, it would be easy. You'd do everything in "local time", record the time updates you receive, timestamped with local time, and figure it all out later. But we need to be able to "schedule" events to occur at predetermined times in the future, so you need some sort of clock model. We also need better than once a day updates. A typical oscillator might be good to, say, 10ppm (variability over the short term), and we need "synchronization" to 1 millisecond. So, in 100 seconds, you'd have drifted out of spec, if you assume a fixed "calibration." The real problem is that the "master" clock might be a crummy 50ppm XO, and all us "slaves" all have TCXOs that are good to a few ppm. It's almost the inverse problem from disciplining a XO with GPS. Jim === > I'm looking for a good (short) description and/or a generalized model for > relating a local time counter to some external reference. Here's the > scenario.. > A computer has a local clock that is a counter being driven by a local > oscillator. > Periodically, we get "time updates" from some outside source that give an > "absolute time" and a sync. > s there. ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
Re: [time-nuts] looking for good description/generalized model for time adjustments
Maybe let the clocks run unadjusted all day and resync between fixed times once a day? This is what the power companies do. You can back-figure the precise times if you really need to from the once a day correction files. This assumes the clock rates are just off a bit, and do not randomly fluctuate. -John === > I'm looking for a good (short) description and/or a generalized model for > relating a local time counter to some external reference. Here's the > scenario.. > A computer has a local clock that is a counter being driven by a local > oscillator. > Periodically, we get "time updates" from some outside source that give an > "absolute time" and a sync. > > We need to use "calibrated time" to record log files and schedule events. > > A simple way to relate local counter to absolute time is: > Calibrated time = (local counter / counter rate estimate) + time offset. > > Calculating the rate estimate from (local counter at time B - local > counter at time A) / (absolute time at time B - absolute time at time A) > And offset in a similar way. > > But this has the problem that the "calibrated time" might have > discontinuities. Especially if you change the rate estimate and offset in > a step function. Calibrated time might even go backwards. > > I guess an additional requirement on "Calibrated time" is that it be > continuous and monotonically increasing. I can see that one can get this > by "gradually" adjusting the rate estimate and offset (i.e. if your local > clock runs fast so your local calibrated time has gotten ahead of absolute > time, you set the rate lower than the true rate until it has slipped back > fast and then slowly bring the rate back to the true rate) > > And this works pretty well if the outside reference is "better" than my > local clock (and is essentially what NTP does, right?), but what if the > reverse is true. > Consider if my local clock is high quality, but I have to discipline > Calibrated Time to an outside time that is of much poorer quality. (By > the gods, it's 6AM when the sun rises and 6PM when the sun sets, so make > everything adjust to fit that, and don't give me grief about seasonal > variations in sunrise/set times). > > What's a good way to do this, hopefully that allows later reconstruction > of what the history of transformations was (e.g. I should be able to take > my calibrated time, and convert it back into local counter time, by using > the log of changes in model parameters). > > Thanks for ideas and pointers in advance > Jim > > > > > > James Lux, P.E. > Task Manager, SOMD Software Defined Radios > Flight Communications Systems Section > Jet Propulsion Laboratory > 4800 Oak Grove Drive, Mail Stop 161-213 > Pasadena, CA, 91109 > +1(818)354-2075 phone > +1(818)393-6875 fax > ___ > time-nuts mailing list -- time-nuts@febo.com > To unsubscribe, go to > https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts > and follow the instructions there. > > ___ time-nuts mailing list -- time-nuts@febo.com To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts and follow the instructions there.
