Hi S.Ancelot, The ETF feature allow the application reserving the packet sending out time. For example, after a procedure completes data processing and requires sending out the result, the procedure can specify to send at the time of middle of next cycle. In order to specify the sending time, the procedure need to set the sending time to packet SO_TXTIME field though CMSG API, and then send the packet though socket.
Because the data procedure can reserve sending time, if the reserving time is far away enough to hold the jitter of the data procedure output time, the isochronism jitter of packet sending time can be very tiny (< 1us). However, you can't have the best of both worlds, the latency will increase. On Xenomai, there are two approaches for above scenario on Xenomai. 1. Assuming the Xenomai core is parasitic in a TSN enabled Linux Distro. Split data procedure into two parts, data processing and data transmission. Make data processing running as Cobalt thread and data transmission running as Linux thread. Both threads share data though share memory. The data transmission adapts ETF mode to send packet. The data processing thread wakes up the data transmission thread after output data is ready. In this approach, the jitter comes from two waking up activities. This approach has been tested. 2. Assuming TSN has been enabled on top of RTnet. The data procedure can complete data processing and transmission after only one waking up. This approach doesn't implement yet. Of course, the jitter of one waking up should be less than two waking up. So comparing with two approaches, although the outcoming packets have the same jitter of isochronism, but latency of the second approach should be less than the first approach. Isochronism and latency, different applications have different emphasis. You can choose you approach according your application. Thanks. Regards, Gong, Xiaoyan From: Stéphane Ancelot <sance...@numalliance.com> Sent: Thursday, June 18, 2020 12:26 AM To: Peter Wong <tsnu...@outlook.com>; Meng, Fino <fino.m...@intel.com>; Jan Kiszka <jan.kis...@siemens.com>; Lange Norbert <norbert.la...@andritz.com>; Xenomai (xenomai@xenomai.org) <xenomai@xenomai.org>; Pirou, Florent <florent.pi...@intel.com>; Gong, Xiaoyan <xiaoyan.g...@intel.com>; Pu, Yanfeng <yanfeng...@intel.com>; Zhang, Wei E <wei.e.zh...@intel.com> Subject: Re: TSN support on xenomai Le 17/06/2020 à 16:59, Peter Wong a écrit : Hi S.Ancelot, You have done some comparison between kernel network stack vs xenomai RTnet stack? Can you share some data ? Giving some data will not help anybody that really depends on your application. Looking at tc - ETF features of the latest kernels , I had to understand a bit how it was working . The most complicated thing to understand is how to setup nic queues. I managed to run our proprietary Ethercat master with RT Preempt with an RT task cycle of 1ms, to sync drives using DC clocks . There can be big jitter of this task without any problem. The only requirement is being sure the next frame programmed being sent BEFORE the next DC sync. (in my case at least 180us before next DC sync) And in this case, offloading and dedicating this task to the i210 NIC hardware helps. Some interesting past weblinks used to understand, it has a lot evolved since ...: https://elinux.org/images/5/56/ELC-2018-USA-TSNonLinux.pdf https://github.com/torvalds/linux/blob/master/net/sched/sch_etf.c Time Based packet transmlission , now renamed ETF : https://lwn.net/Articles/744797/<https://lwn.net/Articles/744797/> v2 : https://lwn.net/Articles/758592/ ... SO_TXTIME: https://gitlab.freedesktop.org/lima/linux/commit/80b14dee2bea128928537d61c333f24cb8cbb62f You can search google for data, with TSN and OPC-UA protocol too. ________________________________ From: Stéphane Ancelot <sance...@numalliance.com><mailto:sance...@numalliance.com> Sent: Wednesday, June 17, 2020 9:19 PM To: Peter Wong <tsnu...@outlook.com><mailto:tsnu...@outlook.com>; Meng, Fino <fino.m...@intel.com><mailto:fino.m...@intel.com>; Jan Kiszka <jan.kis...@siemens.com><mailto:jan.kis...@siemens.com>; Lange Norbert <norbert.la...@andritz.com><mailto:norbert.la...@andritz.com>; Xenomai (xenomai@xenomai.org<mailto:xenomai@xenomai.org>) <xenomai@xenomai.org><mailto:xenomai@xenomai.org>; Pirou, Florent <florent.pi...@intel.com><mailto:florent.pi...@intel.com>; Gong, Xiaoyan <xiaoyan.g...@intel.com><mailto:xiaoyan.g...@intel.com>; Pu, Yanfeng <yanfeng...@intel.com><mailto:yanfeng...@intel.com>; Zhang, Wei E <wei.e.zh...@intel.com><mailto:wei.e.zh...@intel.com> Subject: Re: TSN support on xenomai Le 17/06/2020 à 09:25, Peter Wong a écrit : I think of 2 benefit by running tsn over rtnet + xenomai network driver * better performance since xenomai delivering stringent real-time guarantee and rtnet is smaller network stack * xenomai rt application can run on tsn network Is that right? >From my experience : The answer is no. The kernel network stack has been rewritten in order being efficient to achieve these tasks. Furthermore , thanks to the way TSN (or ethercat) protocol works you don't need a very hard realtime OS, allowing a bigger jitter for the RT task using the network. This is up to the nic hardware component to send the frame at the programmed time . The only RT task requirement is that it should have provided the frame to the nic interfaces before the next clock sync occurs. Regards, S.Ancelot ________________________________ From: Meng, Fino <fino.m...@intel.com><mailto:fino.m...@intel.com> Sent: Wednesday, June 17, 2020 2:08 PM To: Jan Kiszka <jan.kis...@siemens.com><mailto:jan.kis...@siemens.com>; Peter Wong <tsnu...@outlook.com><mailto:tsnu...@outlook.com>; Lange Norbert <norbert.la...@andritz.com><mailto:norbert.la...@andritz.com>; Stéphane Ancelot <sance...@numalliance.com><mailto:sance...@numalliance.com>; Xenomai (xenomai@xenomai.org<mailto:xenomai@xenomai.org>) <xenomai@xenomai.org><mailto:xenomai@xenomai.org>; Pirou, Florent <florent.pi...@intel.com><mailto:florent.pi...@intel.com>; Gong, Xiaoyan <xiaoyan.g...@intel.com><mailto:xiaoyan.g...@intel.com>; Pu, Yanfeng <yanfeng...@intel.com><mailto:yanfeng...@intel.com>; Zhang, Wei E <wei.e.zh...@intel.com><mailto:wei.e.zh...@intel.com> Subject: RE: TSN support on xenomai >Sent: Tuesday, June 16, 2020 10:39 PM > >On 16.06.20 16:00, Peter Wong wrote: >> Can we compile linuxptp (cobalt POSIX wrap) to run on RTnet + Xenomai >> I210 driver so that we only need one NIC to run TSN on RTnet? >> > >There is nothing like that upstream so far. Maybe Fino or Florent >already have some plans, though I expect them to target newer hardware. > >Jan We are working on TSN over Xenomai/Cobalt. For example, have tested such simple case: The synchronizing NIC is inside Linux domain, the TSN stack on vanilla kernel, Cobalt Thread share data to Linux thread first, then send out through TSN stack, Cyclic is 250us, 802.1Qbv enabled, single way, 5 minutes, with some stress-ng workload and 100M noise traffic, test result is identical with preempt-rt Linux, although this case may not match real world scenario. Our current target is find a BKM, let Xenomai app can use the TSN stack, so the synchronizing NIC is stay in vanilla kernel. Let TSN & RTNet merge into one NIC is not in current plan. But if found real world use case really need it, we would like to working on it, BR fino > >> ------------------------------------------------------------------------ >> *From:* Xenomai >> <xenomai-boun...@xenomai.org><mailto:xenomai-boun...@xenomai.org> on behalf >> of Jan Kiszka >> via Xenomai <xenomai@xenomai.org><mailto:xenomai@xenomai.org> >> *Sent:* Tuesday, June 16, 2020 5:44 PM >> *To:* Lange Norbert >> <norbert.la...@andritz.com><mailto:norbert.la...@andritz.com>; Stéphane >> Ancelot >> <sance...@numalliance.com><mailto:sance...@numalliance.com>; Xenomai >> (xenomai@xenomai.org<mailto:xenomai@xenomai.org>) >> <xenomai@xenomai.org><mailto:xenomai@xenomai.org> >> *Subject:* Re: TSN support on xenomai >> >> On 16.06.20 11:41, Lange Norbert wrote: >>> >>> >>>> -----Original Message----- >>>> From: Xenomai >>>> <xenomai-boun...@xenomai.org><mailto:xenomai-boun...@xenomai.org> On >>>> Behalf Of Jan Kiszka >>>> via Xenomai >>>> Sent: Dienstag, 16. Juni 2020 10:58 >>>> To: Stéphane Ancelot >>>> <sance...@numalliance.com><mailto:sance...@numalliance.com>; >>>> xenomai@xenomai.org<mailto:xenomai@xenomai.org> >>>> Subject: Re: TSN support on xenomai >>>> >>>> NON-ANDRITZ SOURCE: BE CAUTIOUS WITH CONTENT, LINKS OR >>>> ATTACHMENTS. >>>> >>>> >>>> On 16.06.20 10:55, Stéphane Ancelot via Xenomai wrote: >>>>> Hi, >>>>> >>>>> This is natively supported in standard linux kernel >>>>> >>>> >>>> Right. What is missing it linking the Xenomai timebase with that of the >>>> kernel >>>> so that we would benefit from Linux doing the sync for us already. >>> >>> But that means you can't use the synchronizing NIC with RTNet / Realtime >>> traffic. >>> >>> Regarding linking timebases, it would help (for various things) >>> If you could easily read those various timebases from the kernel. The rest >>> can happen in >>> userspace. >>> >>> I actually have 3: Linux Monotonic, Xenomai Monotonic, and a PPS >>> synchronized IEEE1588 clock, >>> Cant fold them into one but sometimes need to map between these. >>> Solved this by letting the NIC read out all 3 and providing them to >>> (Xenomai) userspace. >>> >>> Better would be a shared mmap (with 2 or more buffers), so that Linux tasks >>> can asynchronously get that info too. >>> >> >> Latest Intel hardware supports mapping for the PTP time to the system >> time (tsc) natively. You only need to read out the conversion factors in >> the end. That's what we want to have as well. >> >> If using that will create a conflict with using the NIC for RT traffic? >> Likely. But that needs to be sorted out in the future, not only on Intel hw. >> >> Jan >> >> -- >> Siemens AG, Corporate Technology, CT RDA IOT SES-DE >> Corporate Competence Center Embedded Linux >> > >-- >Siemens AG, Corporate Technology, CT RDA IOT SES-DE >Corporate Competence Center Embedded Linux
