Re: [Xenomai] Isolation of CPU (isolcpus=1). Unexpected better performance when RT thread is on core0.

[Philippe Gerum]

On 02/05/2018 04:50 PM, Yann le Chevoir wrote:

Hello,

I am an engineering student and I try to proof that a 4000Hz hard real-time
application can run on an ARM board rather than on a more powerful machine.

I work with an IMX6 dual-core and Xenomai Cobalt 3.0.4. I use POSIX skin.
By the way, I first installed Xenomai Cobalt 3.0.5 but first
experimentations revealed that Alchemy API did not work properly (for
example, altency test did not work).


Any specifics regarding what went wrong would be more helpful. 
Otherwise, nobody may bother and a potential bug would stay.


 I needed more investigations but when

I tried previous version, it worked. I did not test v3.0.6.



You should not have downgraded but rather pulled the latest code from 
the stable-3.0.x branch at git://git.xenomai.org/xenomai-3.git.
As a general note, please disregard the release tarballs: our release 
cycle is way too slow to make them a sane option, as truckloads of bug 
fixes can pass before a new tarball is issued. Tracking the stable tree 
would get you the latest validated fixes.



For now, my point is that I observe some unexpected behaviors when
isolating cpu1 and perhaps you can explain some to me.


TID 881 is the main.
I am not sure why there is the TID 890 thread. Is it a Xenomai one (main)?


libcobalt's internal printer loop thread for carrying out deferred 
printf() calls. Ancillary stuff.



Min execution time is 32us.
Max execution time is 82us.
I am a bit disappointed by so execution-time variations.
How can we explain that?



A dual kernel system exhibits a permanent conflict between two kernels 
competing for the same hw resources. Considering CPU caches for 
instance, the cachelines a sleeping rt thread was previously using can 
be evicted by a non-rt thread resuming on the same CPU then treading on 
a large amount of physical memory. When the rt thread wakes up 
eventually, it may have to go through a series of cache misses to get 
the I/D caches hot again.


Generally speaking, we have a GPOS running side by side a RTOS on the 
same hardware, and the former does not care a dime about the 
requirements of the latter. Mitigating the adverse effects of such 
situation in order to keep latency low and bounded is the basic task 
defining the Xenomai project.


This issue may be aggravated by hw specifics: your imx6d is likely 
fitted with a PL3xx outer L2 cache, for which the write-allocate policy 
is enabled by the kernel. That policy proved to be responsible for ugly 
latency figures with this cache controller. Can we disable such policy? 
Maybe, it depends; we used to have some success doing just that with 
early imx6 hw, then keeping it enabled became a requirement later with 
more recent SoCs (e.g. imx6qp) as we noticed that such policy was 
involved in cache coherence in multi-core configs. So YMMV.


If you want to give WA disabling a try, just pass l2x0_write_allocate=0 
to the kernel cmdline. If your SoC ends up not booting with that switch, 
or dies in mysterious and random ways during runtime, well, this it is 
likely the sign that a cache coherence issue is biting and you can't 
hack away with that one.




Then, trying permutations to understand these variations, I decided to put
thread1 on CPU0. Linux, main, thread0 and dohell continue doing their stuff.
Note that there is again the isolcpus=1 argument, so nothing is on CPU1.
I am surprised to have a better execution time statistics. Is it a known
situation and how can we explain that? See "Core0.png".

Reminder of the configuration when plotting "Core0.png":
Core0: Linux stressed + main + thread0 + thread1
Core1: -

Min execution time is 32us.
Max execution time is 65us.


Then, given these results, as I had the feeling that a mono-core processor
performs better that a dual-core one, I tried to delete the isolcpus=1
argument to proof the contrary.

Here is the configuration when plotting "NoIsolation.png":
Core0: Linux stressed + main + thread0
Core1: Linux stressed + thread1

As you can see, the graph looks like the first one, but execution time
is even worse at 94us.

Is there something I do wrong?


You may also need to tell Xenomai that only CPU1 should process rt 
workloads (i.e. xenomai.supported_cpus=2). I suspect that serialization 
on a core Xenomai lock from all CPUs where the local TWDs tick 
introduces some jitter. Restricting the set of rt CPUs to CPU1 would 
prevent Xenomai from handling rt timer events on any other CPU, lifting 
any contention of that lock in the same move.


--
Philippe.

___
Xenomai mailing list
Xenomai@xenomai.org
https://xenomai.org/mailman/listinfo/xenomai

[Xenomai] Patched kernel headers - are they needed afterwards for gibc and more?

[Lange Norbert]

Hello,

A script with patches together a kernel, ipipe and xenomai kernel drivers. 
Result is the packages to install the kernel image.

A buildroot configuration will build the rootfs, without
  * using the identical kernel version (different patch level)
  * using headers that were used for building the kernel.
  * using the kernel .config

Instead, buildroot DL a different Linux kernel version by itself and used those 
headers.
I don`t think they are needed outside of glibc, but I am not sure.

Further, buildroot will create a "staging" installation for building further 
tools, including those using Xenomai APIs.

So, is this generally safe or something to avoid. I haven't ran into any issues 
yet.

Differences in the basic include files (same kernel version) seem to be rather 
trivial

diff -burN /tmp/header/include/asm-generic/mman-common.h 
/tmp/patched/usr/include/asm-generic/mman-common.h
--- /tmp/header/include/asm-generic/mman-common.h   2018-02-06 
14:26:43.657405976 +0100
+++ /tmp/patched/usr/include/asm-generic/mman-common.h  2018-02-06 
13:43:26.0 +0100
@@ -19,6 +19,9 @@
 #define MAP_TYPE   0x0f/* Mask for type of mapping */
 #define MAP_FIXED  0x10/* Interpret addr exactly */
 #define MAP_ANONYMOUS  0x20/* don't use a file */
+#ifndef MAP_BRK
+# define MAP_BRK   0
+#endif
 #ifdef CONFIG_MMAP_ALLOW_UNINITIALIZED
 # define MAP_UNINITIALIZED 0x400   /* For anonymous mmap, memory could be 
uninitialized */
 #else
diff -burN /tmp/header/include/asm-generic/resource.h 
/tmp/patched/usr/include/asm-generic/resource.h
--- /tmp/header/include/asm-generic/resource.h  2018-02-06 14:26:43.689406277 
+0100
+++ /tmp/patched/usr/include/asm-generic/resource.h 2018-02-06 
13:43:26.0 +0100
@@ -57,5 +57,13 @@
 # define RLIM_INFINITY (~0UL)
 #endif

+/*
+ * Limit the stack by to some sane default: root can always
+ * increase this limit if needed..  8MB seems reasonable.
+ */
+#ifndef _STK_LIM
+# define _STK_LIM  (8*1024*1024)
+#endif
+

 #endif /* _ASM_GENERIC_RESOURCE_H */
diff -burN /tmp/header/include/linux/resource.h 
/tmp/patched/usr/include/linux/resource.h
--- /tmp/header/include/linux/resource.h2018-02-06 14:26:46.433432104 
+0100
+++ /tmp/patched/usr/include/linux/resource.h   2018-02-06 13:43:28.0 
+0100
@@ -59,12 +59,6 @@
 #definePRIO_USER   2

 /*
- * Limit the stack by to some sane default: root can always
- * increase this limit if needed..  8MB seems reasonable.
- */
-#define _STK_LIM   (8*1024*1024)
-
-/*
  * GPG2 wants 64kB of mlocked memory, to make sure pass phrases
  * and other sensitive information are never written to disk.
  */

Kind regards

Norbert Lange
AT-DES
ANDRITZ HYDRO GmbH
Eibesbrunnergasse 20
1120 Wien, Austria
APhone.: +43 50805 56684
. MailTo:norbert.la...@andritz.com
  



This message and any attachments are solely for the use of the intended 
recipients. They may contain privileged and/or confidential information or 
other information protected from disclosure. If you are not an intended 
recipient, you are hereby notified that you received this email in error and 
that any review, dissemination, distribution or copying of this email and any 
attachment is strictly prohibited. If you have received this email in error, 
please contact the sender and delete the message and any attachment from your 
system.

ANDRITZ HYDRO GmbH


Rechtsform/ Legal form: Gesellschaft mit beschränkter Haftung / Corporation

Firmensitz/ Registered seat: Wien

Firmenbuchgericht/ Court of registry: Handelsgericht Wien

Firmenbuchnummer/ Company registration: FN 61833 g

DVR: 0605077

UID-Nr.: ATU14756806


Thank You


___
Xenomai mailing list
Xenomai@xenomai.org
https://xenomai.org/mailman/listinfo/xenomai

Re: [Xenomai] [PATCH] fixes for rpi 2b build

[Philippe Gerum]

On 02/06/2018 05:09 AM, Greg Gallagher wrote:

Small fix ups to enable building multi_v7_defconfig
---
  drivers/gpio/gpio-mvebu.c  |  1 +
  drivers/gpio/gpio-pl061.c  |  1 +
  drivers/gpio/gpio-zynq.c   |  2 +-
  drivers/irqchip/irq-atmel-aic5.c   |  6 --
  drivers/irqchip/irq-bcm7120-l2.c   | 15 +--
  drivers/irqchip/irq-brcmstb-l2.c   | 10 ++
  drivers/irqchip/irq-dw-apb-ictl.c  |  1 +
  drivers/pinctrl/pinctrl-rockchip.c |  8 
  drivers/soc/dove/pmu.c |  1 +
  9 files changed, 28 insertions(+), 17 deletions(-)



Merged, thanks.

--
Philippe.

___
Xenomai mailing list
Xenomai@xenomai.org
https://xenomai.org/mailman/listinfo/xenomai