Re: Recommendation on NTP appliances/devices
Quoting Julien Goodwin : Show my anything short of a classic SONET transmission system (or perhaps sync-E) where you actually have something with jitter that low [tens of microseconds]. Since you asked, here you go: http://i.imgur.com/DvMJd5y.png Two EndRun Unison GPS NTP servers, one in New York metro and one in London metro. They're connected via 10G over dark fiber (along with a bunch of networking gear doing more than just measuring time). Measurements taken from ntp running on the New York server, the blue line is the offset between the two clocks (in ms, left labels) and the pink line is the rtt (in ms, right labels). 90th percentile of the offsets is 34 microseconds, and 10th percentile is 5 microseconds. You can see a spike in one-way latency near sample "590". Most likely culprit is of one of the firewalls being busy (there's two in this path).
Re: Recommendation on NTP appliances/devices
> So what, that sends IP packets, are you using to *measure* it. I can Agilent if we need unidir. Normal run-of-the-mill 10GE SP router will give you low single digit microsecond jitter when not congested. (You can run 99.99% no problem, as long as you don't try >100% (i.e. >1 interface sending)). We only do bidir for constant measurements of network, unidir is unfortunately only for troubleshooting case-by-case. It would be very nice to always have unidir view to the network, but cost/benefit is not there if you have lot of pops. -- ++ytti
Re: Recommendation on NTP appliances/devices
On 04/04/14 21:48, Saku Ytti wrote: > On (2014-04-04 20:37 +1100), Julien Goodwin wrote: > >>> Meinberg[0] pegs rubidium at ±8ms per year, if you need NTP to do say single >>> direction backbone SLA measurement you want to have microsecond precision. >> >> Those two statements don't go together. > > Point I was making is that free-running rubidium is not accurate enough for > QoS measurements of IP core. Free running oscillators are fine as long as you know what the actual specs are (and have the unit measured to that). >> Also outside the HFTers most of us don't care about a few milliseconds >> (sure an extra 50ms can be a pain, but is trivial to measure). > > Jitter in backbone is low tens of microseconds, if you want to measure how > that changes over time, free-running rubidium is not going to cut it. What you'd actually measure is a side affect of buffer depth at any point. Show my anything short of a classic SONET transmission system (or perhaps sync-E) where you actually have something with jitter that low. So what, that sends IP packets, are you using to *measure* it. I can imagine using an FPGA hard clocked to a reference oscillator (and even a TCXO is good enough) doing it, but I'm not aware of any actual implementation of this. Again, short of the HFT world I just can't imagine how this could actually matter.
Re: Recommendation on NTP appliances/devices
On (2014-04-04 20:37 +1100), Julien Goodwin wrote: > > Meinberg[0] pegs rubidium at ±8ms per year, if you need NTP to do say single > > direction backbone SLA measurement you want to have microsecond precision. > > Those two statements don't go together. Point I was making is that free-running rubidium is not accurate enough for QoS measurements of IP core. > Also outside the HFTers most of us don't care about a few milliseconds > (sure an extra 50ms can be a pain, but is trivial to measure). Jitter in backbone is low tens of microseconds, if you want to measure how that changes over time, free-running rubidium is not going to cut it. -- ++ytti
Re: Recommendation on NTP appliances/devices
On Thu, 3 Apr 2014, David Hubbard wrote: Anyone have recommendations on NTP appliances; i.e. make, model, gps vs cell, etc.? Roof/outdoor/window access not available. Would ideally need to be able to handle bursts of up to a few thousand simultaneous queries. Needs IPv6 support. For some diversity you could try: - WWVB/CHU radio with a good indoor antenna into an appliance - CDMA, which yes is based on GPS, but tied with Rb oscillator can carry over any reception outages (CDMA or GPS) - Of course just setup an NTP server that peers to pool.ntp.org (but perphaps the least desirable) I've seen good results using the Endrun CDMA units as well as the WWVB units, both appliances and IPv6-enabled. Symmetricom does this too. wfms
Re: Recommendation on NTP appliances/devices
On 04/04/14 10:16, Majdi S. Abbas wrote: > On Thu, Apr 03, 2014 at 06:55:02PM -0400, David Hubbard wrote: >> Anyone have recommendations on NTP appliances; i.e. make, model, gps vs >> cell, etc.? Roof/outdoor/window access not available. Would ideally >> need to be able to handle bursts of up to a few thousand simultaneous >> queries. Needs IPv6 support. > > Without roof access I'd suggest CDMA instead of GPS: > > http://www.endruntechnologies.com/ntp-server.htm > > Appears to fit your requirements. > > --msa > The downside of CDMA is it's going to live until Verizon & Sprint can get enough of their customers migrated to LTE. It really depends on how accurate you need to be. If you only want <10ms accuracy but stable (It's trivial to get all clients better than 1ms) then grab three to five old servers (or new low-power ones), and just put ntpd on them, pointing at some nearby upstreams. If it *must* be an appliance the Symmetricom units are nice, and support IPv6 (have done for years).
Re: Recommendation on NTP appliances/devices
On 04/04/14 17:29, Saku Ytti wrote: > On (2014-04-03 21:25 -0700), Will Orton wrote: > >> There are commercially available NTP servers with GPS + Rb oscillators... >> for NTP >> use you could basically let it sync up a couple days, disconnect the GPS and >> let >> it freerun. You'd still be within a millisecond of GPS even after a couple >> years >> most likely. Reconnect it to GPS for a couple days every 1-2 years to resync >> it. >> More fun and cheaper to build your own I'd bet, if you had the time. > > Meinberg[0] pegs rubidium at ±8ms per year, if you need NTP to do say single > direction backbone SLA measurement you want to have microsecond precision. Those two statements don't go together. Also outside the HFTers most of us don't care about a few milliseconds (sure an extra 50ms can be a pain, but is trivial to measure). It's one thing to be a time-nut (I'm certainly one), but recognise that straight NTP, well deployed, even syncing from the pool is sufficient for nearly all use cases not needing sub-millisecond precision. > I think most GPS chipsets today do Glonass also, maybe it's partly because > Russia has import sanctions for those who don't do, or maybe simply because it > gives better user experience as sync is found earlier. > > But is there NTP product which you can configure to GPS-only mode or > Glonass-only mode, which I think might be closer to Rob's idea of redundancy. > As if you use both, 'poisoned' source would break all of your kit. > But that is probably not easy to solve with two sources, if half is GPS and > half is Glonass, and Glonass starts sending bad timing information, what do > your NTP clients do, average to the middle? > > [0] http://www.meinbergglobal.com/english/specs/gpsopt.htm >
Re: Recommendation on NTP appliances/devices
On (2014-04-03 21:25 -0700), Will Orton wrote: > There are commercially available NTP servers with GPS + Rb oscillators... for > NTP > use you could basically let it sync up a couple days, disconnect the GPS and > let > it freerun. You'd still be within a millisecond of GPS even after a couple > years > most likely. Reconnect it to GPS for a couple days every 1-2 years to resync > it. > More fun and cheaper to build your own I'd bet, if you had the time. Meinberg[0] pegs rubidium at ±8ms per year, if you need NTP to do say single direction backbone SLA measurement you want to have microsecond precision. I think most GPS chipsets today do Glonass also, maybe it's partly because Russia has import sanctions for those who don't do, or maybe simply because it gives better user experience as sync is found earlier. But is there NTP product which you can configure to GPS-only mode or Glonass-only mode, which I think might be closer to Rob's idea of redundancy. As if you use both, 'poisoned' source would break all of your kit. But that is probably not easy to solve with two sources, if half is GPS and half is Glonass, and Glonass starts sending bad timing information, what do your NTP clients do, average to the middle? [0] http://www.meinbergglobal.com/english/specs/gpsopt.htm -- ++ytti
Re: Recommendation on NTP appliances/devices
On Thu, Apr 03, 2014 at 09:06:57PM -0700, George Herbert wrote: > Sadly, right now that either means your own real clock, or WWV. The > cellphone time is (as far as I know, for the networks I saw data on) all > coming off GPS. > > Fortunately real clocks are coming way down in cost. There are commercially available NTP servers with GPS + Rb oscillators... for NTP use you could basically let it sync up a couple days, disconnect the GPS and let it freerun. You'd still be within a millisecond of GPS even after a couple years most likely. Reconnect it to GPS for a couple days every 1-2 years to resync it. More fun and cheaper to build your own I'd bet, if you had the time. With clocks/oscillators designed to provide hold-over for synchronous networks and microwave RF systems (parts per million or billion) the demands of NTP for general use in an IP network are pretty modest. You lose more accuracy in NTP stratum 1->2 across a (relaively) jittery WAN link than a cheap atomic clock does in a long time. -Will
Re: Recommendation on NTP appliances/devices
Loves my old Heathkit WWVB unit. Keeps drift in check most days. Pairs nicely with the Spectracom 9383. Looking at the Microsemi TP-5000 w/ rubidium oscillator. /bill On Thu, Apr 03, 2014 at 10:25:07PM -0400, Rob Seastrom wrote: > > On a tangential note, it's all very nice to say "We have brand X and > like them", but I'd be curious to hear from folks who have deployed at > least four divergent brands with non-overlapping GPS chip sets and > software [*] to keep a conspiracy of errors from causing the time to > suddenly be massively incorrect. Not that this has ever happened in > the past in a single vendor configuration [cough]. > > Along the same lines I'm troubled by the lack of divergent sources > these days - everything seems slaved to GPS either directly or > indirectly (might be nice to have stuff out there that got its time > exclusively via Galileo or Glonass). The sole exception that I can > think of offhand is that I have an office within ground wave of WWVB, > which would be a tasty ingredient. GOES is gone. LORAN is defunded. > And so it goes; all our eggs are in one basket. > > I've thought about posting this request to the NTP developers list, > but maybe someone who's an operator and actually cares about keeping > the byzantine generals sequestered from each other has solved this > problem recently. > > Clues? > > -r > > > [*] to the extent possible; I'm sure that there's a lot of reference > implementation DNA floating around out there) > > > Berry Mobley writes: > > > We have symmetricom (now microsemi) and are very happy with them, but we > > use the roof mounted gps antennas. They will peer with public ntp severs if > > that would work for you. > > > > David Hubbard wrote: > > > >>Anyone have recommendations on NTP appliances; i.e. make, model, gps vs > >>cell, etc.? Roof/outdoor/window access not available. Would ideally > >>need to be able to handle bursts of up to a few thousand simultaneous > >>queries. Needs IPv6 support. > >> > >>Thanks! > >>
Re: Recommendation on NTP appliances/devices
On Thu, Apr 3, 2014 at 8:46 PM, Rob Seastrom wrote: > > Chris Adams writes: > > > Once upon a time, Rob Seastrom said: > >> Along the same lines I'm troubled by the lack of divergent sources > >> these days - everything seems slaved to GPS either directly or > >> indirectly (might be nice to have stuff out there that got its time > >> exclusively via Galileo or Glonass). > > > > Since you mentioned GLONASS: it had a 10+ hour outage yesterday, > > apparently due to a bad ephemeris upload. Did anybody have a > > GLONASS-using NTP server experience problems? > > It would be the height of arrogance to think that this couldn't happen to > GPS. > > I want redundancy. > Sadly, right now that either means your own real clock, or WWV. The cellphone time is (as far as I know, for the networks I saw data on) all coming off GPS. Fortunately real clocks are coming way down in cost. So the question is, if you want redundancy, what do your failure modes look like. Is some low level drift if GPS goes away and stays away for an extended period OK? In that case, redundancy probably would be a single local high grade clock. Do you want multi-vendor-common-mode-failure-resistant low drift if GPS goes away? In that case, you probably need 3 local clocks. Possibly 4, if you want to be able to down one for maintenance and still have 3 operating when the fit hits the shan, so that if one of the remaining ones drifts you know which of the 3 is out of whack and to exclude from the "live source". Just two operating and you're SOL on figuring out which one is off. This is why spacecraft and aircraft often have 3 or 4 of each critical thing; 3 gets you "only fly with all 3 working" and the ability to detect the bad instrument; 4 lets you fly with one down for maintenance and still have safe redundant operation, increasing dispatch reliability. -- -george william herbert george.herb...@gmail.com
Re: Recommendation on NTP appliances/devices
Chris Adams writes: > Once upon a time, Rob Seastrom said: >> Along the same lines I'm troubled by the lack of divergent sources >> these days - everything seems slaved to GPS either directly or >> indirectly (might be nice to have stuff out there that got its time >> exclusively via Galileo or Glonass). > > Since you mentioned GLONASS: it had a 10+ hour outage yesterday, > apparently due to a bad ephemeris upload. Did anybody have a > GLONASS-using NTP server experience problems? It would be the height of arrogance to think that this couldn't happen to GPS. I want redundancy. -r
Re: Recommendation on NTP appliances/devices
Once upon a time, Rob Seastrom said: > Along the same lines I'm troubled by the lack of divergent sources > these days - everything seems slaved to GPS either directly or > indirectly (might be nice to have stuff out there that got its time > exclusively via Galileo or Glonass). Since you mentioned GLONASS: it had a 10+ hour outage yesterday, apparently due to a bad ephemeris upload. Did anybody have a GLONASS-using NTP server experience problems? -- Chris Adams
Re: Recommendation on NTP appliances/devices
On a tangential note, it's all very nice to say "We have brand X and like them", but I'd be curious to hear from folks who have deployed at least four divergent brands with non-overlapping GPS chip sets and software [*] to keep a conspiracy of errors from causing the time to suddenly be massively incorrect. Not that this has ever happened in the past in a single vendor configuration [cough]. Along the same lines I'm troubled by the lack of divergent sources these days - everything seems slaved to GPS either directly or indirectly (might be nice to have stuff out there that got its time exclusively via Galileo or Glonass). The sole exception that I can think of offhand is that I have an office within ground wave of WWVB, which would be a tasty ingredient. GOES is gone. LORAN is defunded. And so it goes; all our eggs are in one basket. I've thought about posting this request to the NTP developers list, but maybe someone who's an operator and actually cares about keeping the byzantine generals sequestered from each other has solved this problem recently. Clues? -r [*] to the extent possible; I'm sure that there's a lot of reference implementation DNA floating around out there) Berry Mobley writes: > We have symmetricom (now microsemi) and are very happy with them, but we use > the roof mounted gps antennas. They will peer with public ntp severs if that > would work for you. > > David Hubbard wrote: > >>Anyone have recommendations on NTP appliances; i.e. make, model, gps vs >>cell, etc.? Roof/outdoor/window access not available. Would ideally >>need to be able to handle bursts of up to a few thousand simultaneous >>queries. Needs IPv6 support. >> >>Thanks! >>
Re: Recommendation on NTP appliances/devices
We have symmetricom (now microsemi) and are very happy with them, but we use the roof mounted gps antennas. They will peer with public ntp severs if that would work for you. David Hubbard wrote: >Anyone have recommendations on NTP appliances; i.e. make, model, gps vs >cell, etc.? Roof/outdoor/window access not available. Would ideally >need to be able to handle bursts of up to a few thousand simultaneous >queries. Needs IPv6 support. > >Thanks! >
Re: Recommendation on NTP appliances/devices
On Thu, Apr 03, 2014 at 06:55:02PM -0400, David Hubbard wrote: > Anyone have recommendations on NTP appliances; i.e. make, model, gps vs > cell, etc.? Roof/outdoor/window access not available. Would ideally > need to be able to handle bursts of up to a few thousand simultaneous > queries. Needs IPv6 support. Without roof access I'd suggest CDMA instead of GPS: http://www.endruntechnologies.com/ntp-server.htm Appears to fit your requirements. --msa