[Linuxptp-devel] linuxptp filter architure
Richard, I'm doing some experiments with linuxptp. It's kind of a neat gadget to have abound, thanks. I'd like to try adding lucky packet filters. I think these might be useful to get to improved accuracy given residual delay variations in the on-path support's timestamping. The experiments also include timestamping and timekeeping mechanisms, but for now I wanted to talk about how to integrate the filters. I'm having a problem fitting suitable filters into the current architecture. There is a disconnect between my understanding of the problem and how the filter class fits into the overall structure. The details of the disconnect are as follows: In Ptp, the upstream and downstream flows provide delay measurements. If PDV makes only 10% of the measurements useful, it is the filter's job to find these few to use for updating the offset estimate. The two flows are statistically independent. This says that although the individual odds of finding a useful measurement is 10%, the joint odds of finding a pair of good measurements is 10% squared or 1%. This says that you get to keep many more useful measurements if the upstream and downstream experiment streams are filtered separately. To support separate filtering and give the filter access to everything it needs, I think what I would like to see is the following refactoring and see how it works out: Change the argument list for filter_create to add an indication of which filter is being created, and pointers to the clock_type and config. Change the argument list for filter_sample to be local time of the experiment and observed delay. Return value would be filtered delay. Where delay is rxtimestamp - txtimestamp with corrections. Change port.c to create two filters and use them on the one-way measurements before calling clock_synchronize and clock_path_delay. The goal is to support improved filtering without breaking all that is currently supported. I think it wise to leave in the current single filter for the combined RTT with a config flag to switch the new calling locations. Any interest in such a path? If so, not sure what the mechanics you prefer. -Stuart ___ Linuxptp-devel mailing list Linuxptp-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linuxptp-devel
Re: [Linuxptp-devel] linuxptp filter architure
On Fri, Sep 20, 2019 at 04:39:06PM +, Stuart Venters wrote: > Change the argument list for filter_create to add an indication of > which filter is being created, and pointers to the clock_type and > config. > Change the argument list for filter_sample to be local time of the > experiment and observed delay. Return value would be filtered > delay. Where delay is rxtimestamp - txtimestamp with corrections. No idea why "local time of the experiment" should matter, but I'll wait to see the code. > Change port.c to create two filters and use them on the one-way > measurements before calling clock_synchronize and clock_path_delay. The tsproc API already differentiates upstream versus downstream, and so it should be easy for you to add a new tsproc_mode that keeps your two "lucky" filters. Why not do this... Change tsproc_create() to tsproc_create(struct config *cfg, const char *portname, enum tsproc_mode mode); and then allocate one or two filters depending on 'mode' ? > Any interest in such a path? Yes, indeed! Tell us please whether the above will work. Thanks, Richard ___ Linuxptp-devel mailing list Linuxptp-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linuxptp-devel
Re: [Linuxptp-devel] linuxptp filter architure
On Fri, Sep 20, 2019 at 04:39:06PM +, Stuart Venters wrote: > To support separate filtering and give the filter access to everything it > needs, I think what I would like to see is the following refactoring and see > how it works out: > > Change the argument list for filter_create to add an indication of which > filter is being created, and pointers to the clock_type and config. > > Change the argument list for filter_sample to be local time of the experiment > and observed delay. Return value would be filtered delay. Where delay is > rxtimestamp - txtimestamp with corrections. > > Change port.c to create two filters and use them on the one-way measurements > before calling clock_synchronize and clock_path_delay. Why port.c? As Richard suggested, tsproc might be a better place. It would help if you could describe your approach in a bit more detail. Is the minimum delay configurable or estimated? How do you prevent it from getting stuck on rapid frequency changes? To me it sounds like you want to drop the measurements as if the packets were lost, i.e. something different than what the filter.h API is intended for. That wouldn't work well with the current servo architecture. Reusing old measurements might be easier, but I suspect it could be tricky. -- Miroslav Lichvar ___ Linuxptp-devel mailing list Linuxptp-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linuxptp-devel
Re: [Linuxptp-devel] linuxptp filter architure
Richard, Miroslav, Thanks for the response. Yes definitely, tsproc instead of port.c, not sure what I was thinking. I'm glad I asked. I'm trying to figure out how to fit an experiment into LinuxPtp and end up with as much as possible worth pushing back up. It looks like the dual filters are easy to fit in. Let me talk about the rest of the story and see if you have any other ideas. The parts talked about are a Timekeeping framework, filtering framework, packet selection, and timestamping. For timekeeping I'm drawn to history in the way ship's navigators used the chronometer. They never adjusted it and instead just kept notes on what they learned about it's error. That way if something they learned turned out to be in error, they could sort it out later. For PTP, I wish to do the same. Keep the local clock steady, unaffected by the servo. Keep a history of measurements based on the local clock. This gives ability to rethink previous decisions as new information comes in. This client's job is then to find the offset to add to local time to make the reconstructed master's time. Depending on the local clock, this offset may converge to a fixed number, or (as you point out) vary in interesting ways. To describe the filtering framework, let me paint a picture I like to call a time tunnel because it has a hole in the middle we are steering the offset through. Given this local reference, one can calculate an apparent offset for each measurement assuming zero transport delay. (For PTP these are the corrected t1-t2 and t4-t3.) If you do a scatter plot of these versus local time, you will get a clump for each direction, one above and one below the actual offset. The scatter plot combines stats from the local clock offset variations and the pdv in each direction. The distance between the clumps is the RTT. The center of the space between the clumps is the offset. Given this, you can sketch two curves, picking out the statistic you desire (perhaps average or lowest delay times) for the packets in each direction. Then draw a curve half way between these as the measured offset. Then use this to drive a servo to construct the smoothed, estimated offset. The packet selectors implementing the above sketch depend on the local reference. Without SyncE, the local reference will vary in phase and frequency. But, there are still bounds on the how much it can vary in the short term depending on the spec for the local oscillator. These can be used to propagate expanding error bounds on measurements forward and backwards in time. (This is why I'd like to have the local time along with the measurement.) These can be used to let good measurements outweigh measurements which contain no useful information. This gives a starting point for picking measurements. You can add more complex bounds mechanisms which track how the local reference is moving if you wish. With SyncE to condition the local clock, the offset is not time varying (a flat line). So the constructed bounds are also flat, but maybe still decreasing weight after a while to favor newer measurements. (Since I'm interested in higher accuracy, this is the use case I'm focused on for now.) For timestamping, I have hardware which looks like a half of a TC function. It has a local clock which counts time since boot in the correction field format. (Clocked from the Ethernet RX clock for SyncE.) On packet receive, it picks out the event packets and subtracts the local clock from the correction field in the packet. (Incrementally updating the udp and fcs.) On packet transmit, it picks out the event packets and adds the local clock to the correction field. It's a half TC function because packet switching hardware supporting TC might implement these functions so a packet transiting the box encounters both the receive and transmit functions. I plan to do the other half in the PTP packet handling software. S/W rx reads local time and both sets the rx timestamp to this value and adds it to the correction field. S/W tx reads local time and both sets the tx timestamp and subtracts same from the correction field. Given this, no special per-packet timestamping functions are required in the kernel and the reading of local time does not need to be especially accurate. For output, I can tell the hardware what local time to put out a 1PPS for testing. Not particularly interested in synchronizing the Linux system time to the ptp time. The above seems a mind shift from where things are today, so I'd like to first do something easy to see if it works before making it pretty. I'd like to stay out of kernel space for now to keep the scope of the experiment small enough to actually get done. (Open /dev/mem and map a window to the fpga regs into the ptp4l.) Given this, I can add another timestamping mode and put the s/w timestamping into bc_event and port_prepare_and_send. You have pointed out where to put in the dual filters. The servo looks
Re: [Linuxptp-devel] linuxptp filter architure
Very nice read. But some points: The rethinking of old-decisions w.r.t. new info may not be valid as the time to apply them is long gone since the conditions under which those decisions were made. Maybe the local clock was actually leading or lagging behind at that time, which might not be the case any more as seen in the new info. Consuming packets that show a lower than average delay PDV is a good technique, but even that applies to the current window/instant, can it be applied in retrospect to re-evaluate somewhat older offset calculation? The scatter plot is scattered because of the local clock variation w.r.t. the master clock's stability and the n/w, kernel, time-stamping effect etc. Thus, the weighted average that gives importance to more recent data-points is more valuable (after ofcourse filtering our some rogue packets). Also, why assume the former, that it would ever converge to a fixed number unless the slave clock is at-least as stable as the master with lesser drift and frequency variation due to internal crystal characteristics and temprature etc, in which case it should be the master. Isnt it? Mabe it is still good and wise to have the local-clock (un-corrected) available, but if we have SyncE, and the local clock is at-least corrected for the frequency aspect w.r.t. master, the phase variation can then converge to smaller offsets and corections. On Tue, Sep 24, 2019 at 7:38 AM Stuart Venters wrote: > Richard, > Miroslav, > > Thanks for the response. > > Yes definitely, tsproc instead of port.c, not sure what I was thinking. > I'm glad I asked. > > I'm trying to figure out how to fit an experiment into LinuxPtp and end up > with as much as possible worth pushing back up. > It looks like the dual filters are easy to fit in. > Let me talk about the rest of the story and see if you have any other > ideas. > > The parts talked about are a Timekeeping framework, filtering framework, > packet selection, and timestamping. > > For timekeeping I'm drawn to history in the way ship's navigators used the > chronometer. > They never adjusted it and instead just kept notes on what they learned > about it's error. That way if something they learned turned out to be in > error, they could sort it out later. > For PTP, I wish to do the same. Keep the local clock steady, unaffected > by the servo. > Keep a history of measurements based on the local clock. This gives > ability to rethink previous decisions as new information comes in. > This client's job is then to find the offset to add to local time to make > the reconstructed master's time. > Depending on the local clock, this offset may converge to a fixed number, > or (as you point out) vary in interesting ways. > > To describe the filtering framework, let me paint a picture I like to call > a time tunnel because it has a hole in the middle we are steering the > offset through. > Given this local reference, one can calculate an apparent offset for each > measurement assuming zero transport delay. (For PTP these are the corrected > t1-t2 and t4-t3.) > If you do a scatter plot of these versus local time, you will get a clump > for each direction, one above and one below the actual offset. > The scatter plot combines stats from the local clock offset variations and > the pdv in each direction. > The distance between the clumps is the RTT. The center of the space > between the clumps is the offset. > Given this, you can sketch two curves, picking out the statistic you > desire (perhaps average or lowest delay times) for the packets in each > direction. > Then draw a curve half way between these as the measured offset. > Then use this to drive a servo to construct the smoothed, estimated offset. > > The packet selectors implementing the above sketch depend on the local > reference. > Without SyncE, the local reference will vary in phase and frequency. > But, there are still bounds on the how much it can vary in the short term > depending on the spec for the local oscillator. > These can be used to propagate expanding error bounds on measurements > forward and backwards in time. > (This is why I'd like to have the local time along with the measurement.) > These can be used to let good measurements outweigh measurements which > contain no useful information. > This gives a starting point for picking measurements. > You can add more complex bounds mechanisms which track how the local > reference is moving if you wish. > > With SyncE to condition the local clock, the offset is not time varying (a > flat line). > So the constructed bounds are also flat, but maybe still decreasing weight > after a while to favor newer measurements. > (Since I'm interested in higher accuracy, this is the use case I'm focused > on for now.) > > For timestamping, I have hardware which looks like a half of a TC function. > It has a local clock which counts time since boot in the correction field > format. (Clocked from the Ethernet RX clock for SyncE.) > On packet receive, i
Re: [Linuxptp-devel] linuxptp filter architure
On Mon, Sep 23, 2019 at 08:31:30PM +, Stuart Venters wrote: > The remaining part appears to be figuring out how to turn off the all the > kernel packet timestamping help. > Also, turning off the /dev/ptp0 stuff. > > > Thoughts? Sounds like a great step backwards. Really. I didn't quite follow the explanation of your correction field counter thing, but I can only recommend this. - Make a proper PHC for your fpga. - Implement the SO_TIMESTAMPING API for your fpga. - Use the modular tsproc and servo in linuxptp for your filters. Good luck, Richard ___ Linuxptp-devel mailing list Linuxptp-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linuxptp-devel
Re: [Linuxptp-devel] linuxptp filter architure
On Mon, Sep 23, 2019 at 08:31:30PM +, Stuart Venters wrote: > I'm trying to figure out how to fit an experiment into LinuxPtp and end up > with as much as possible worth pushing back up. > It looks like the dual filters are easy to fit in. > Let me talk about the rest of the story and see if you have any other ideas. The high-level overview makes sense to me. It reminds me of the TICSync algorithm. I suspect it won't fit the current linuxptp design very well. I'm not sure how it would be useful with SyncE. I'd suggest to hack something quick together first, but support the existing HW timestamping API, so we can test how it performs. The implementation in linuxptp can be figured out later. -- Miroslav Lichvar ___ Linuxptp-devel mailing list Linuxptp-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linuxptp-devel
