Martin,
It doesn't make a lot of sense to moo time based on a herd of TSU
counters unless each cow in the herd is wrangled to a common timescale.
The fastest machine I have, a Sun Blade 1500, can moo time in about 400
nanoseconds. If each cow in the herd has a frequency error up to 100 PPM
and the accuracy expectation is one microsecond, the moo needs to be
adjusted on the order of 100 times each second. This requires a
processor interrupt at intervals of 10 ms and an adjustment for each cow
computed for convenience at intervals of one second.
I think you see where I am going. The infrastructure is just like
wrangling time in a SMP system where each processor has its own cycle
counter and where the counter can be read at any time from any
processor. This infrastructure has been implemented in the stock kernel
of the Digital/HP Alpha running Tru64.
There are lots of ways to milk the cow, but some ways may be more
practical than others. All need to reveal the four timestamps T1-T4 in
one form or another. Your method has the advantage of compatability, but
hazards the overhead of preamble, header, checksum and guard time, as
well as the possibility of collision and retransmission.
In my favorite the frame format leaves space for two timestamps
following the checksum. An outgoing timestamp is struck at the first bit
following the preamble and transmitted twice after the checksum. An
incoming timestamp is struck the same way and copied after the
timestamps in the receive buffer. The timestamps are rejected if the
first two don't match. This scheme remains accurate even if the frame is
retransmitted and takes advantage of the inherent resistance of the NTP
algorithms to data omission.
A disadvantage with either method is that the useful capability in the
stock NTP algorithm to detect replayed and spoofed packets is
diminished, depending on how the driver timestamps are revealed to the
core algorithms.
Dave
Martin Burnicki wrote:
Dave,
David L. Mills wrote:
Martin,
I don't think 1588 makes a lot of sense over UDP unless the NIC has a
TSU built in.
Agreed.
The net (pardon the pun) result is very much like the old
line discipline/streams gadget we used to use on serial lines when the
Sun IPC driver stack had jitter over 23 ms. It's easy to timestamp on
the way in as you describe. The hard part is to timestamp on the way
out; the driver needs a feature where NTP or whatever can ask for the
identifier and timestmap of the packet just sent. The sneaky way to do
this is for the driver to copy this information in the header of the
next packet received.
If the time stamp unit is capable of listening on the interconnection lines
between the PHY and the MAC of the network board, it can take timestamps on
incoming and outgoing packets in the same way, with the same accuracy.
Finding a way for a driver to get the time stamps out of the TSU and passing
it up to application is just a matter of specifications and policy. The
proble which arises is that the time stamp of an outgoing packet is only
available after the packet has already slipped onto the wire. This time
stamp is an additional piece of information which can't be sent in the
packet itself. Also, modification of the outgoing packet on the hardware
level would make crypto hashes invalid and unusable.
PTP has an elegant method of sending a second packet which contains the time
stamp of the initial outgoing packet.
I don't see serious problems with jitter; algorithms like the NTP clock
filter and/or median filter make short work of that, unless the network
has serious congestion and frequent retransmissions. I'm getting a few
microseconds jitter with a PPS signal, DCD interface and 60-s median
filter now. I would hope a gigabit Ethernet with TSU and driver could
shave that to 10-100 ns if collisions and/or buffering is minimal.
Currently there are no TSU solutions I'm aware of for Gigabit Ethernet. The
problem is that the programmable logic devices which run at the required
speed are very expensive and power consuming.
I assume the TSU counters, system timer and CPU clock are not
synchronized, so something has to be done in the application to
compensate for the expected frequency errors.
In Winterthur we've had lots of discussions on this topic with Frank Kardel
who has also attended the plug-fest.
There are different possible approaches to this, each of which naturally has
advantages and disadvantages.
Some operating systems (I think at least some *BSD versions, and upcoming
Linux kernels) have the time-of-day clock strictly separated from its time
base. Those systems could relatively simply be configured to use a TSU's
oscillator as the time base for the system clock, so NTP or PTP just had to
adjust the system time, and all time stamps taken in the TSU would be
related directly to the system time, not to a different clock domain.
[...]
So, where can I buy a NIC with a TSU?
I'm aware of a source for PCI NICs with TSU ...
Martin
_______________________________________________
questions mailing list
[email protected]
https://lists.ntp.isc.org/mailman/listinfo/questions
