Hi Matthias
The e1000e Tx/Rx IRQ threads must have a priority that is higher than
the priority of the realtime thread (the one that calls
ecrt_send/receive) and all throttling done by the e1000e must be
disabled. The priority of the Ethercat-OP thread is not really a problem.
But, we had a problem on some e1000e revisions when a network statistic
watch was running (e.g. gnome-system-monitor)
> watch -n 0.1 cat /sys/class/net/eth1/statistics/*
Use the kernel tracers:
Start trace and after a few seconds, hit Ctrl-C:
> sudo trace-cmd record -e sched -e 'irq_handler_*' -p function -l
'ec_*' -l 'ecrt_*' -l 'e1000*' -b 16384 -s 250 -r 90
Watch trace (use thread filters and zoom)
> kernelshark
BTW. our machines use a 250microsecond cycle time!
Regards
Martin
On 01.02.2016 14:01, Dr.-Ing. Matthias Schöpfer wrote:
Hi Armin,
thanks for your message. We are using this kernel version in some other
realtime setups successfully. cyclictest and hwlatdetect show, that the
system is running as one would expect.
In these setups, we usually got a hardware interrupt, and it was usually
necessary to boost the priority of the threaded interrupt to be totally
safe even in situations of high (I/O) load.
So, my question is, if it could be a fruitful endeavor to look into the
etherlab kernel modules to set the priority for ethercat related
tasks/threads. From looking from outside, I see a EtherCAT-OP thread,
which I can easily put to RT prio (and did), but it seems that most work
is done in the ksoftirqd, which as far as I know is of general use.
I have not looked into the sources, and I am not 100% sure if this is
the source of some problems we see here. Therefore my question, has
anyone looked into it? If it is just about dispatching the work to a
dedicated kernel thread instead of ksoftirqd the required changes would
possibly small...
Thanks and Regards,
Mattthias
On 02/01/2016 11:14 AM, Armin Steinhoff wrote:
Hi Dr. Schöpfer,
please have a look to the test rack at the OASDL homepage and you will
see that the same release of PREEMPT_RT is performing very different on
different motherboards.
IMHO ... the most important issue is the SMI interrupt and the triggered
execution of the low level firmware of the motherboard.
Also a transaction on the PCI bus using DMA transfers can increase the
response time on a single core ...
Best Regards
Armin
http://www.steinhoff-automation.com
Dr.-Ing. Matthias Schöpfer schrieb:
Hi Thomas Winding,
thanks for your answer!
We are running kernel 3.12.31-rt45, since the newer kernel versions do
not seem to run as reliable regarding real time at least on our hardware.
ethercat is from the mecurical repository and we use the e1000e driver.
Most of our problems we might have solved, since we had an issue with
our cycle time / calculations where sometimes, our cycle lasted 600-800
microseconds, which seems to be way to long. We are now down to < 220
microseconds.
In rare cases, when we start the software, we still constantly running
in the mentioned issues. When we stop and restart, everything is fine. I
wonder, if we sometimes hit a misfortuned timing. As of lately, I was
not able to reproduce it.
Regarding your suspicion: I wonder, if it is better to sync the
transmits to the 1 ms cycle instead of the receives?!
Regards,
Matthias Schöpfer
On 02/01/2016 09:07 AM, Thomas Winding wrote:
Hi Matthias Schöpfer
I have seen the same problem. I am also running at 1 ms and using
clock_nanosleep.
I suspect that the problem arise when you send a new telegram
before you have received the previously send telegram.
Which version of the ethercat are you using?
Which version of kernel are you using?
Best regard,
Thomas Winding
-----Original Message-----
From: etherlab-dev [mailto:etherlab-dev-boun...@etherlab.org] On
Behalf Of Dr.-Ing. Matthias Schöpfer
Sent: 26. januar 2016 14:22
To: etherlab-dev@etherlab.org
Subject: [etherlab-dev] Possible Realtime Issues with Ethercat Master
and RT Preempt Kernel
Hi!
We started using etherlab/ethercat and are quite impressed. Nice work!
We are running Linux with a RT_PREEMPT Kernel and e1000e ethercat
driver. We have to run at a cycle time of 1ms and we have jitter from
clock_nanosleep of about 15 microsecs max.
Nevertheless, we suffer from time to time from these:
EtherCAT WARNING 0: 2 datagrams UNMATCHED!
EtherCAT 0: Domain 0: Working counter changed to 9/9.
EtherCAT 0: Domain 0: Working counter changed to 0/9.
Especially, when we apply load to the system. From previous projects,
I experienced these effects when IRQ/Kernel Thread was not set to
appropriate RT Level.
My Question: has anybody experienced similar problems, and would it
be worth to investigate it. And if I decide to patch the kernel
module, where is a good starting point.
Thanks and regards,
Matthias Schöpfer
--
Dr. Matthias Schöpfer
mz robolab GmbH
Marie-Curie-Str. 1
53359 Rheinbach
Office: +49 2226 83600 00
Fax: +49 2226 83600 11
Email: schoep...@robolab.de
mz robolab GmbH
Vertretungsberechtigte GeschÀftsfÃŒhrer: Dr. RÃŒdiger MaaÃ, Ralf
Schulte Registergericht Amtsgericht Bonn Registernummer HRB 10595
---
This e-mail may contain confidential and/or privileged information.
If you are not the intended recipient (or have received this e-mail in
error) please notify the sender immediately and destroy this e-mail.
Any unauthorised copying, disclosure or distribution of the material
in this e-mail is strictly forbidden.
Diese E-Mail enthÀlt vertrauliche und/oder rechtlich geschÌtzte
Informationen. Wenn Sie nicht der richtige Adressat sind oder diese
E-Mail irrtÃŒmlich erhalten haben, informieren Sie bitte sofort den
Absender und vernichten Sie diese Mail. Das unerlaubte Kopieren sowie
die unbefugte Weitergabe dieser Mail ist nicht gestattet.
---
P please consider the environment before printing this e-mail
_______________________________________________
etherlab-dev mailing list
etherlab-dev@etherlab.org
http://lists.etherlab.org/mailman/listinfo/etherlab-dev
Note:
This e-mail is for the named person's use only. It may contain confidential
and/or privileged information. If you have received this e-mail in error,
please notify the sender immediately and delete the material from any system.
Any unauthorized copying, disclosure, distribution or other use of this
information by persons or entities other than the intended recipient is
prohibited.
Thank You.
_______________________________________________
etherlab-dev mailing list
etherlab-dev@etherlab.org
http://lists.etherlab.org/mailman/listinfo/etherlab-dev
