Re: libcopperplate.so depends on modecheck?

[Jan Kiszka via Xenomai]

On 31.05.19 17:55, Philippe Gerum wrote:

On 5/31/19 5:31 PM, Jan Kiszka wrote:

On 31.05.19 17:13, Lange Norbert wrote:

-Original Message-
From: Jan Kiszka 
Sent: Freitag, 31. Mai 2019 16:26
To: Lange Norbert ; Xenomai
(xenomai@xenomai.org) ; Philippe Gerum
; Henning Schild 
Subject: Re: libcopperplate.so depends on modecheck?

E-MAIL FROM A NON-ANDRITZ SOURCE: AS A SECURITY MEASURE, PLEASE
EXERCISE CAUTION WITH E-MAIL CONTENT AND ANY LINKS OR
ATTACHMENTS.


On 31.05.19 14:12, Lange Norbert via Xenomai wrote:

I still believe this to be a bug, adding a patch to correct this.



The patch lacks a commit message that explains "why". Also, inline
would be
preferred (unless your mail client is consistently preventing that).


I have the choice between git-send-email without context of the
previous mail,
or Outlook which breaks any formatting.
(I was kinda expecting a discussion and thus a V2)


Something which starts with "[PATCH]" and continues with code has a
higher chance of attention, but your questions would have deserved
feedback nevertheless.




The idea of the wrapping is to establish an instrumentation point
that will
give us an early warning when an application calls into non-rt
malloc/free
from rt context. As those glibc calls only occasionally cause a
syscall, the hook
is needed to have that deterministic warning.


Sure, but this is NOT __wrap_malloc but __real_malloc, which seems
pointless to me
(this is just routed through as-is to malloc).



As I wrote, something is fishy for me as well. But we should first try
to understand why the author added those tags (original intention) and
then match it against the current implementation.


The original authors says that you can drop the __STD() tag from all
call sites in libcopperplate/*.c, the intent was to prevent mode
checkers to get in the way when such calls are legit would a rt context
is actually running non-rt stuff like prep or cleanup code. However,
regular pthreads or Xenomai threads running the weak scheduling policy
are the only possible callers, so we basically don't care.



OK, perfect. Norbert, please update your commit log accordingly and send as 
inline patch.


Thanks,
Jan

--
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: libcopperplate.so depends on modecheck?

[Philippe Gerum via Xenomai]

On 5/31/19 5:31 PM, Jan Kiszka wrote:
> On 31.05.19 17:13, Lange Norbert wrote:
>>> -Original Message-
>>> From: Jan Kiszka 
>>> Sent: Freitag, 31. Mai 2019 16:26
>>> To: Lange Norbert ; Xenomai
>>> (xenomai@xenomai.org) ; Philippe Gerum
>>> ; Henning Schild 
>>> Subject: Re: libcopperplate.so depends on modecheck?
>>>
>>> E-MAIL FROM A NON-ANDRITZ SOURCE: AS A SECURITY MEASURE, PLEASE
>>> EXERCISE CAUTION WITH E-MAIL CONTENT AND ANY LINKS OR
>>> ATTACHMENTS.
>>>
>>>
>>> On 31.05.19 14:12, Lange Norbert via Xenomai wrote:
>>>> I still believe this to be a bug, adding a patch to correct this.
>>>>
>>>
>>> The patch lacks a commit message that explains "why". Also, inline
>>> would be
>>> preferred (unless your mail client is consistently preventing that).
>>
>> I have the choice between git-send-email without context of the
>> previous mail,
>> or Outlook which breaks any formatting.
>> (I was kinda expecting a discussion and thus a V2)
> 
> Something which starts with "[PATCH]" and continues with code has a
> higher chance of attention, but your questions would have deserved
> feedback nevertheless.
> 
>>
>>> The idea of the wrapping is to establish an instrumentation point
>>> that will
>>> give us an early warning when an application calls into non-rt
>>> malloc/free
>>> from rt context. As those glibc calls only occasionally cause a
>>> syscall, the hook
>>> is needed to have that deterministic warning.
>>
>> Sure, but this is NOT __wrap_malloc but __real_malloc, which seems
>> pointless to me
>> (this is just routed through as-is to malloc).
>>
> 
> As I wrote, something is fishy for me as well. But we should first try
> to understand why the author added those tags (original intention) and
> then match it against the current implementation.

The original authors says that you can drop the __STD() tag from all
call sites in libcopperplate/*.c, the intent was to prevent mode
checkers to get in the way when such calls are legit would a rt context
is actually running non-rt stuff like prep or cleanup code. However,
regular pthreads or Xenomai threads running the weak scheduling policy
are the only possible callers, so we basically don't care.

-- 
Philippe.

Re: libcopperplate.so depends on modecheck?

[Jan Kiszka via Xenomai]

On 31.05.19 17:13, Lange Norbert wrote:

-Original Message-
From: Jan Kiszka 
Sent: Freitag, 31. Mai 2019 16:26
To: Lange Norbert ; Xenomai
(xenomai@xenomai.org) ; Philippe Gerum
; Henning Schild 
Subject: Re: libcopperplate.so depends on modecheck?

E-MAIL FROM A NON-ANDRITZ SOURCE: AS A SECURITY MEASURE, PLEASE
EXERCISE CAUTION WITH E-MAIL CONTENT AND ANY LINKS OR
ATTACHMENTS.


On 31.05.19 14:12, Lange Norbert via Xenomai wrote:

I still believe this to be a bug, adding a patch to correct this.



The patch lacks a commit message that explains "why". Also, inline would be
preferred (unless your mail client is consistently preventing that).


I have the choice between git-send-email without context of the previous mail,
or Outlook which breaks any formatting.
(I was kinda expecting a discussion and thus a V2)


Something which starts with "[PATCH]" and continues with code has a higher 
chance of attention, but your questions would have deserved feedback nevertheless.





The idea of the wrapping is to establish an instrumentation point that will
give us an early warning when an application calls into non-rt malloc/free
from rt context. As those glibc calls only occasionally cause a syscall, the 
hook
is needed to have that deterministic warning.


Sure, but this is NOT __wrap_malloc but __real_malloc, which seems pointless to 
me
(this is just routed through as-is to malloc).



As I wrote, something is fishy for me as well. But we should first try to 
understand why the author added those tags (original intention) and then match 
it against the current implementation.



The bigger idea behind this is to use preload libraries (#1), at which point 
this symbol wrapping is
counter-productive.



Then please propose tool and method for upstream integration if it can replace 
existing static instrumentations without drawbacks.


Jan



That instrumentation should also work for indirect malloc/free via our
libraries. From that perspective, I would rather expect unwrapped calls so
that they *do* point to libmodechk helpers. Seems inconsistent from that
angle. OTH, internal calls could also be instrumented at build-time, thus could
avoid the wrapping trick.


Again, the symbols are not the checked ones by libmodechk, and you can avoid the
build-time wrapping with preloaded libraries.


But all this is fairly complex. Philippe and Henning should have thought this
through a bit more.

Jan


#1: https://github.com/nolange/preload_checkers

Regards,
Norbert



Regards, Norbert


-Original Message-
From: Lange Norbert
Sent: Mittwoch, 20. März 2019 11:49
To: Xenomai (xenomai@xenomai.org) 
Subject: libcopperplate.so depends on modecheck?

Hello,

Seems like libcopperplate.so always depends on libmodechk.so, as the
following symbols would imply:

nm -D /home/lano/buildroot/host/x86_64-buildroot-linux-
gnu/sysroot/usr/xenomai/lib/libcopperplate.so | grep __real
   U __real_free
   U __real_malloc

xeno-config deals with this by always linking modeck, even if you add
--no-
mode-check-

I am not completely confident with the wrapper-magic, but shouldn't
libcopperplate use either cobalt_malloc or the libc malloc?
__real_malloc is just a stub for libc malloc, so I really don't get
the point (assuming this is intentional).

Modeswitches could also be detected by interposing libraries, so
removing libmodechk Is a viable thing to do.
(shameless plug: https://github.com/nolange/preload_checkers )


Regards, Norbert



This message and any attachments are solely for the use of the intended

recipients. They may contain privileged and/or confidential information or
other information protected from disclosure. If you are not an intended
recipient, you are hereby notified that you received this email in error and
that any review, dissemination, distribution or copying of this email and any
attachment is strictly prohibited. If you have received this email in error,
please contact the sender and delete the message and any attachment from
your system.


ANDRITZ HYDRO GmbH


Rechtsform/ Legal form: Gesellschaft mit beschränkter Haftung /
Corporation

Firmensitz/ Registered seat: Wien

Firmenbuchgericht/ Court of registry: Handelsgericht Wien

Firmenbuchnummer/ Company registration: FN 61833 g

DVR: 0605077

UID-Nr.: ATU14756806


Thank You

-- next part -- A non-text attachment was
scrubbed...
Name: 0001-remove-__STD-for-heap-function.patch
Type: application/octet-stream
Size: 5265 bytes
Desc: 0001-remove-__STD-for-heap-function.patch
URL:


<http://xenomai.org/pipermail/xenomai/attachments/20190531/6c808169/a
t

tachment.obj>


--
Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate
Competence Center Embedded Linux



This message and any attachments are solely for the use of the intended 
recipients. They may co

RE: libcopperplate.so depends on modecheck?

[Lange Norbert via Xenomai]

> -Original Message-
> From: Jan Kiszka 
> Sent: Freitag, 31. Mai 2019 16:26
> To: Lange Norbert ; Xenomai
> (xenomai@xenomai.org) ; Philippe Gerum
> ; Henning Schild 
> Subject: Re: libcopperplate.so depends on modecheck?
>
> E-MAIL FROM A NON-ANDRITZ SOURCE: AS A SECURITY MEASURE, PLEASE
> EXERCISE CAUTION WITH E-MAIL CONTENT AND ANY LINKS OR
> ATTACHMENTS.
>
>
> On 31.05.19 14:12, Lange Norbert via Xenomai wrote:
> > I still believe this to be a bug, adding a patch to correct this.
> >
>
> The patch lacks a commit message that explains "why". Also, inline would be
> preferred (unless your mail client is consistently preventing that).

I have the choice between git-send-email without context of the previous mail,
or Outlook which breaks any formatting.
(I was kinda expecting a discussion and thus a V2)

> The idea of the wrapping is to establish an instrumentation point that will
> give us an early warning when an application calls into non-rt malloc/free
> from rt context. As those glibc calls only occasionally cause a syscall, the 
> hook
> is needed to have that deterministic warning.

Sure, but this is NOT __wrap_malloc but __real_malloc, which seems pointless to 
me
(this is just routed through as-is to malloc).

The bigger idea behind this is to use preload libraries (#1), at which point 
this symbol wrapping is
counter-productive.

>
> That instrumentation should also work for indirect malloc/free via our
> libraries. From that perspective, I would rather expect unwrapped calls so
> that they *do* point to libmodechk helpers. Seems inconsistent from that
> angle. OTH, internal calls could also be instrumented at build-time, thus 
> could
> avoid the wrapping trick.

Again, the symbols are not the checked ones by libmodechk, and you can avoid the
build-time wrapping with preloaded libraries.

> But all this is fairly complex. Philippe and Henning should have thought this
> through a bit more.
>
> Jan

#1: https://github.com/nolange/preload_checkers

Regards,
Norbert
>
> > Regards, Norbert
> >
> >> -Original Message-
> >> From: Lange Norbert
> >> Sent: Mittwoch, 20. März 2019 11:49
> >> To: Xenomai (xenomai@xenomai.org) 
> >> Subject: libcopperplate.so depends on modecheck?
> >>
> >> Hello,
> >>
> >> Seems like libcopperplate.so always depends on libmodechk.so, as the
> >> following symbols would imply:
> >>
> >> nm -D /home/lano/buildroot/host/x86_64-buildroot-linux-
> >> gnu/sysroot/usr/xenomai/lib/libcopperplate.so | grep __real
> >>   U __real_free
> >>   U __real_malloc
> >>
> >> xeno-config deals with this by always linking modeck, even if you add
> >> --no-
> >> mode-check-
> >>
> >> I am not completely confident with the wrapper-magic, but shouldn't
> >> libcopperplate use either cobalt_malloc or the libc malloc?
> >> __real_malloc is just a stub for libc malloc, so I really don't get
> >> the point (assuming this is intentional).
> >>
> >> Modeswitches could also be detected by interposing libraries, so
> >> removing libmodechk Is a viable thing to do.
> >> (shameless plug: https://github.com/nolange/preload_checkers )
> >>
> >>
> >> Regards, Norbert
> > 
> >
> > This message and any attachments are solely for the use of the intended
> recipients. They may contain privileged and/or confidential information or
> other information protected from disclosure. If you are not an intended
> recipient, you are hereby notified that you received this email in error and
> that any review, dissemination, distribution or copying of this email and any
> attachment is strictly prohibited. If you have received this email in error,
> please contact the sender and delete the message and any attachment from
> your system.
> >
> > ANDRITZ HYDRO GmbH
> >
> >
> > Rechtsform/ Legal form: Gesellschaft mit beschränkter Haftung /
> > Corporation
> >
> > Firmensitz/ Registered seat: Wien
> >
> > Firmenbuchgericht/ Court of registry: Handelsgericht Wien
> >
> > Firmenbuchnummer/ Company registration: FN 61833 g
> >
> > DVR: 0605077
> >
> > UID-Nr.: ATU14756806
> >
> >
> > Thank You
> > 
> > -- next part -- A non-text attachment was
> > scrubbed...
> > Name: 0001-remove-__STD-for-heap-function.patch
> > Type: application/octet-stream
> > Size: 5265 bytes
> > Desc:

Re: libcopperplate.so depends on modecheck?

[Jan Kiszka via Xenomai]

On 31.05.19 14:12, Lange Norbert via Xenomai wrote:

I still believe this to be a bug, adding a patch to correct this.



The patch lacks a commit message that explains "why". Also, inline would be 
preferred (unless your mail client is consistently preventing that).



The idea of the wrapping is to establish an instrumentation point that will give 
us an early warning when an application calls into non-rt malloc/free from rt 
context. As those glibc calls only occasionally cause a syscall, the hook is 
needed to have that deterministic warning.


That instrumentation should also work for indirect malloc/free via our 
libraries. From that perspective, I would rather expect unwrapped calls so that 
they *do* point to libmodechk helpers. Seems inconsistent from that angle. OTH, 
internal calls could also be instrumented at build-time, thus could avoid the 
wrapping trick.


But all this is fairly complex. Philippe and Henning should have thought this 
through a bit more.


Jan


Regards, Norbert


-Original Message-
From: Lange Norbert
Sent: Mittwoch, 20. März 2019 11:49
To: Xenomai (xenomai@xenomai.org) 
Subject: libcopperplate.so depends on modecheck?

Hello,

Seems like libcopperplate.so always depends on libmodechk.so, as the
following symbols would imply:

nm -D /home/lano/buildroot/host/x86_64-buildroot-linux-
gnu/sysroot/usr/xenomai/lib/libcopperplate.so | grep __real
  U __real_free
  U __real_malloc

xeno-config deals with this by always linking modeck, even if you add --no-
mode-check-

I am not completely confident with the wrapper-magic, but shouldn't
libcopperplate use either cobalt_malloc or the libc malloc?
__real_malloc is just a stub for libc malloc, so I really don't get the point
(assuming this is intentional).

Modeswitches could also be detected by interposing libraries, so removing
libmodechk Is a viable thing to do.
(shameless plug: https://github.com/nolange/preload_checkers )


Regards, Norbert



This message and any attachments are solely for the use of the intended 
recipients. They may contain privileged and/or confidential information or 
other information protected from disclosure. If you are not an intended 
recipient, you are hereby notified that you received this email in error and 
that any review, dissemination, distribution or copying of this email and any 
attachment is strictly prohibited. If you have received this email in error, 
please contact the sender and delete the message and any attachment from your 
system.

ANDRITZ HYDRO GmbH


Rechtsform/ Legal form: Gesellschaft mit beschränkter Haftung / Corporation

Firmensitz/ Registered seat: Wien

Firmenbuchgericht/ Court of registry: Handelsgericht Wien

Firmenbuchnummer/ Company registration: FN 61833 g

DVR: 0605077

UID-Nr.: ATU14756806


Thank You

-- next part --
A non-text attachment was scrubbed...
Name: 0001-remove-__STD-for-heap-function.patch
Type: application/octet-stream
Size: 5265 bytes
Desc: 0001-remove-__STD-for-heap-function.patch
URL: 



--
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

RE: libcopperplate.so depends on modecheck?

[Lange Norbert via Xenomai]

I still believe this to be a bug, adding a patch to correct this.

Regards, Norbert

> -Original Message-
> From: Lange Norbert
> Sent: Mittwoch, 20. März 2019 11:49
> To: Xenomai (xenomai@xenomai.org) 
> Subject: libcopperplate.so depends on modecheck?
>
> Hello,
>
> Seems like libcopperplate.so always depends on libmodechk.so, as the
> following symbols would imply:
>
> nm -D /home/lano/buildroot/host/x86_64-buildroot-linux-
> gnu/sysroot/usr/xenomai/lib/libcopperplate.so | grep __real
>  U __real_free
>  U __real_malloc
>
> xeno-config deals with this by always linking modeck, even if you add --no-
> mode-check-
>
> I am not completely confident with the wrapper-magic, but shouldn't
> libcopperplate use either cobalt_malloc or the libc malloc?
> __real_malloc is just a stub for libc malloc, so I really don't get the point
> (assuming this is intentional).
>
> Modeswitches could also be detected by interposing libraries, so removing
> libmodechk Is a viable thing to do.
> (shameless plug: https://github.com/nolange/preload_checkers )
>
>
> Regards, Norbert


This message and any attachments are solely for the use of the intended 
recipients. They may contain privileged and/or confidential information or 
other information protected from disclosure. If you are not an intended 
recipient, you are hereby notified that you received this email in error and 
that any review, dissemination, distribution or copying of this email and any 
attachment is strictly prohibited. If you have received this email in error, 
please contact the sender and delete the message and any attachment from your 
system.

ANDRITZ HYDRO GmbH


Rechtsform/ Legal form: Gesellschaft mit beschränkter Haftung / Corporation

Firmensitz/ Registered seat: Wien

Firmenbuchgericht/ Court of registry: Handelsgericht Wien

Firmenbuchnummer/ Company registration: FN 61833 g

DVR: 0605077

UID-Nr.: ATU14756806


Thank You

-- next part --
A non-text attachment was scrubbed...
Name: 0001-remove-__STD-for-heap-function.patch
Type: application/octet-stream
Size: 5265 bytes
Desc: 0001-remove-__STD-for-heap-function.patch
URL: