Hello Thomas,
These 13/50 are max latency numbers? both systems would qualify as reasonably
good wrt. rt-preempt.
What I find surprising somehow: even your 'good' system shows so much variance:
min: 6789, max: 50345
Shouldn't that be much always the same? Maybe something else is happening and
amplified on the 'bad' system.
Regards,
Matthieu
On 3/8/16, 7:47 PM, "Thomas Bitsky Jr" <t...@automateddesign.com> wrote:
Matthieu,
[snip]
in other words, you are positive the difference starts with ethercat master?
[/snip]
Yes. I have measured the time around that code block. That is where the
variance occurs that causes me to have unmatched packets.
That being said, one machine (i3) has a 13 microsecond latency, the other
(celeron) has a 50 microsecond latency. If the ethercat userspace API is
“sleeping” my process, then that would probably explain the problem.
Are these problems avoided or reduced in the kernel space, or will I have the
exact same timing?
Thanks!
On 3/8/16, 9:32 PM, "Matthieu Bec" <m...@gmto.org> wrote:
>Hello Thomas,
>
>Both systems give similar cyclictest results? in other words, you are positive
>the difference starts with ethercat master?
>I'm not clear for example if celeron and i3 fall under the same processor
>family when you compile your kernel. Still such difference, explanation would
>seem far fetched.
>
>Regards,
>Matthieu
>
>
>
>
>On 3/8/16, 8:09 AM, "etherlab-users on behalf of Thomas Bitsky Jr"
><etherlab-users-boun...@etherlab.org on behalf of t...@automateddesign.com>
>wrote:
>
>Richard,
>
>I put a lot of time into your suggestions the past few days, and here is where
>I’m at.
>
>[snip]
>we use CLOCK_MONOTONIC for ecrt_master_application_time(), in fact we
>use it thoughout. Probably not the solution, but worth a try.
>[/snip]
>
>
>As I generally do; thanks for catching that error for me.
>
>[snip]
>Once your cyclic test is stable (~10us max jitter), you may start
>looking for delays in your code.
>[/snip]
>
>
>
>Semaphore and mutex calls were waking up inconsistently (the rt tests reported
>the same thing). I re-designed the API to eliminate the need for them in my
>code. I also reduced the memory footprint. Send, recv and timespec functions
>are the only system calls remaining.
>
>This brought the rest of the program down to very little execution time. I
>went through function by function, but couldn’t find the reason for the long
>execution times in my cleaned up code.
>
>However, I still have a surprisingly long execution time. The functions that
>cause the problem are from the ethercat master library.
>
>
>ecrt_master_receive(master_);
>
>
>
> ecrt_domain_process(lrwDomainMgr_.domain);
> ecrt_domain_process(noLrwWriteDomainMgr_.domain);
> ecrt_domain_process(noLrwReadDomainMgr_.domain);
>
>
>
>I have tried this on two pieces of hardware. They run the same operating
>system and configuration, same EtherLAB master version, same Ethernet driver.
>
>System 1 (Intel i3 NUC, 1 Intel e1000e Ethernet port): exec min 6789 .. max
>50345 (nano seconds)
>System 2 (Intel quad celeoron, 4 x e1000e Ethernet port): exec min 15588 … max
>323376 (nano seconds)
>(Both systems: Ubuntu Server 14.04LTS, Linux core 3.12.50-rt68 #1 SMP PREEMPT
>RT)
>
>On system 1, the unmatched packets never happen; for whatever reason, those
>functions are returning much more rapidly. On the second system, the worst
>case happens about once per minute, once every two minutes.
>
>1. Am I unreasonable in thinking these times are excessive? I understand that
>the system is communicating out to external hardware and there will be a
>delay, but the difference between the two computers is surprising to me.
>2. Is this a consequence of using the EtherCAT master from user space? I’ve
>always used it from kernel space before. However, I’ve never done as extensive
>of timing measurements before. The EtherLAB documentation mentioned that the
>userspace library only added 1 microsecond of overhead, if I recall. Is this
>as good as it gets from the user space?
>3. Is there a way to address this problem, or is it completely dependent upon
>the hardware?
>
>
>Thanks for any insights to the problem, and for all the advice everyone has
>submitted this far.
>
>
>
>
>
>
>
>On 3/3/16, 1:59 PM, "Richard Hacker" <h...@igh.de> wrote:
>
>>Hi,
>>
>>we use CLOCK_MONOTONIC for ecrt_master_application_time(), in fact we
>>use it thoughout. Probably not the solution, but worth a try.
>>
>>- Richard
>>
>>On 03.03.2016 17:08, Thomas Bitsky Jr wrote:
>>>
>>> [snip]
>>> Are you using the dc_clock_adjust to modify your sleep time? ie.
>>> something like
>>> [/snip]
>>>
>>>
>>> I’ve been trying, but the results are all over the place and I’m obviously
>>> not understanding what needs to be done with the clock.
>>>
>>> My latest attempt is has the scan time ticking from anywhere between 3930
>>> and 4100 scans per second (reported by ethercat master command) and does
>>> nothing to eliminate skipped datagrams. I don’t think I’m fundamentally
>>> understanding what needs to be done.
>>>
>>> My latest code is below. If anyone has any insights, it would be greatly
>>> appreciated.
>>>
>>> ========
>>>
>>> int64_t dc_adjust_ns = 0 ;
>>> int32_t dc_diff_ns = 0;
>>>
>>>
>>>
>>> void calculateClockDrift(void) {
>>> uint32_t ref_time = 0;
>>> uint64_t prev_app_time = dc_time_ns;
>>>
>>> dc_time_ns = system_time_ns();
>>> // get reference clock time to synchronize master cycle
>>> ecrt_master_reference_clock_time(master_, & ref_time);
>>> dc_diff_ns = (uint32_t) prev_app_time - ref_time;
>>> // calc drift (via un-normalised time diff)
>>> int32_t delta = dc_diff_ns - prev_dc_diff_ns;
>>> prev_dc_diff_ns = dc_diff_ns;
>>>
>>>
>>> dc_diff_ns =
>>> ((dc_diff_ns + (cycle_ns_ / 2)) % cycle_ns_) - (cycle_ns_ / 2);
>>>
>>> }
>>>
>>>
>>> #define TIMESPEC_ADD_NS(TS, NS)\
>>> (TS).tv_nsec += (NS);\
>>> while ((TS).tv_nsec >= NANOS_PER_SEC) {\
>>> (TS).tv_nsec -= NANOS_PER_SEC;\
>>> (TS).tv_sec++;
>>> }
>>>
>>> #define TIMESPEC2NSEPOCH2000(T)\
>>> ((uint64_t)(((T).tv_sec - 946684800 ULL) * 1000000000 ULL) +
>>> (T).tv_nsec)
>>>
>>> # define TON struct timespec# define TON_ENDTIME(MS)\
>>> time_add_ns((MS) * NANOS_PER_MILLISEC)
>>>
>>>
>>> static TON clockSyncTon_;
>>>
>>>
>>> int
>>> TON_ISDONE(struct timespec ts) {
>>> struct timespec now;
>>> clock_gettime(CLOCK_MONOTONIC, & now);
>>> if (now.tv_sec > ts.tv_sec)
>>> return 1;
>>> else if (now.tv_sec == ts.tv_sec && now.tv_nsec >= ts.tv_nsec)
>>> return 1;
>>> else
>>> return 0;
>>> }
>>>
>>>
>>> static bool
>>> wait_period(RtaiMain * inst) {
>>>
>>> int rc;
>>> bool done = false;
>>> while (!done && inst - > doScan && runAll_) {
>>> rc = clock_nanosleep(CLOCK_MONOTONIC,
>>> TIMER_ABSTIME, & inst - > wakeupTime,
>>> NULL);
>>>
>>>
>>> if (rc == EFAULT) {
>>> return false;
>>> } else if (rc == EINTR) {
>>> continue;
>>> } else if (rc == EINVAL) {
>>> return false;
>>> } else {
>>> done = 1;
>>> }
>>> }
>>> TIMESPEC_ADD_NS(inst->wakeupTime,
>>> inst->cycleNs + dc_diff_ns );
>>> return true;
>>>
>>> }
>>>
>>>
>>> static void
>>> cyclic_task(RtaiMain * inst) {
>>>
>>> clock_gettime(CLOCK_MONOTONIC, & (inst - > wakeupTime));
>>> /* start after one second */
>>> inst - > wakeupTime.tv_sec++;
>>> wait_period(inst);
>>> while (runAll_ && inst - > doScan) {
>>> //
>>> // Trigger Fieldbus RX here.
>>> //
>>> //
>>> ecrt_master_receive(master_);
>>>
>>> // record the time we received the data so other parts of the
>>> program
>>> // have an accurate time reading
>>> globalTickTimeNs = ton_get_ns();
>>>
>>> ecrt_domain_process(lrwDomainMgr_.domain);
>>> ecrt_domain_process(noLrwWriteDomainMgr_.domain);
>>> ecrt_domain_process(noLrwReadDomainMgr_.domain);
>>>
>>> if (counter_) {
>>>
>>> counter_—;
>>> } else {
>>> counter_ = 4000;
>>>
>>> // check for master state
>>> check_master_state();
>>> }
>>>
>>>
>>> …
>>> tick sub systems
>>>
>>>
>>> //
>>> // Trigger Fieldbus TX. This should be the last step
>>> //
>>> //
>>> ecrt_domain_queue(lrwDomainMgr_.domain);
>>> ecrt_domain_queue(noLrwWriteDomainMgr_.domain);
>>> ecrt_domain_queue(noLrwReadDomainMgr_.domain);
>>> clock_gettime(CLOCK_REALTIME, & dcTime_);
>>> ecrt_master_application_time(
>>> master_,
>>> TIMESPEC2NSEPOCH2000(dcTime_));
>>>
>>>
>>> if (TON_ISDONE(clockSyncTon_)) {
>>> ecrt_master_sync_reference_clock(master_);
>>> clockSyncTon_ = TON_ENDTIME(10); // milliseconds
>>> }
>>> ecrt_master_sync_slave_clocks(master_);
>>>
>>> // send EtherCAT data
>>> ecrt_master_send(master_);
>>>
>>> calculateClockDrift();
>>>
>>>
>>> if (!wait_period(inst)) {
>>> PRINT("%s Error with waiting! Stopping cyclic_task.\n",
>>> __FUNCTION__);
>>> inst - > doScan = false;
>>> }
>>> }
>>>
>>> }
>>>
>
_______________________________________________
etherlab-users mailing list
etherlab-users@etherlab.org
http://lists.etherlab.org/mailman/listinfo/etherlab-users
