[OE-core] [PATCH 2/5] valgrind: do not strip the package or ptests

2019-05-13 Thread Randy MacLeod 
About half the ptests will fail if the executables deployed
as part of the ptest package are stripped. Currently
there is no easy way to only strip the main valgrind
package so leave it and the ptests *all* unstripped.

There is an enhancement filed to enable wildcard stripping:
   https://bugzilla.yoctoproject.org/show_bug.cgi?id=13343
so this recipe can be fixed once that is implemented.

Signed-off-by: Randy MacLeod 
---
 meta/recipes-devtools/valgrind/valgrind_3.15.0.bb | 4 
 1 file changed, 4 insertions(+)

diff --git a/meta/recipes-devtools/valgrind/valgrind_3.15.0.bb 
b/meta/recipes-devtools/valgrind/valgrind_3.15.0.bb
index 9749fdb244..b72f4d76ee 100644
--- a/meta/recipes-devtools/valgrind/valgrind_3.15.0.bb
+++ b/meta/recipes-devtools/valgrind/valgrind_3.15.0.bb
@@ -114,6 +114,10 @@ RDEPENDS_${PN}-ptest_append_libc-glibc = " glibc-utils"
 # One of the tests contains a bogus interpreter path on purpose.
 # Skip file dependency check
 SKIP_FILEDEPS_${PN}-ptest = '1'
+# INHIBIT_PACKAGE_STRIP_${PN}-ptest = '1'
+# PACKAGE_DEBUG_SPLIT_STYLE_${PN}-ptest = 'debug-file-directory'
+INHIBIT_PACKAGE_STRIP = '1'
+PACKAGE_DEBUG_SPLIT_STYLE = 'debug-file-directory'
 
 do_compile_ptest() {
 oe_runmake check
-- 
2.17.0

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/5] valgrind: do not strip the package or ptests

2019-05-14 Thread Richard Purdie 
On Mon, 2019-05-13 at 23:57 -0400, Randy MacLeod wrote:
> About half the ptests will fail if the executables deployed
> as part of the ptest package are stripped. Currently
> there is no easy way to only strip the main valgrind
> package so leave it and the ptests *all* unstripped.
> 
> There is an enhancement filed to enable wildcard stripping:
>https://bugzilla.yoctoproject.org/show_bug.cgi?id=13343
> so this recipe can be fixed once that is implemented.
> 
> Signed-off-by: Randy MacLeod 
> ---
>  meta/recipes-devtools/valgrind/valgrind_3.15.0.bb | 4 
>  1 file changed, 4 insertions(+)

I did test whether we could get away with a dependency on valgrind-dbg
as in theory that should behave the same way. It doesn't which means
the debug symbol linkage isn't being honoured by valgrind and that is
something we need to look into too :/.

I'm torn on whether to accept this patch or try and fix partial
stripping...

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/5] valgrind: do not strip the package or ptests

2019-05-14 Thread Burton, Ross 
On Mon, 13 May 2019 at 20:57, Randy MacLeod  wrote:
> About half the ptests will fail if the executables deployed
> as part of the ptest package are stripped. Currently
> there is no easy way to only strip the main valgrind
> package so leave it and the ptests *all* unstripped.

What's the disk usage impact of an unstripped valgrind package?

Ross
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/5] valgrind: do not strip the package or ptests

2019-05-14 Thread Randy MacLeod 

On 5/14/19 10:08 AM, Richard Purdie wrote:

On Mon, 2019-05-13 at 23:57 -0400, Randy MacLeod wrote:

About half the ptests will fail if the executables deployed
as part of the ptest package are stripped. Currently
there is no easy way to only strip the main valgrind
package so leave it and the ptests *all* unstripped.

There is an enhancement filed to enable wildcard stripping:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=13343
so this recipe can be fixed once that is implemented.

Signed-off-by: Randy MacLeod 
---
  meta/recipes-devtools/valgrind/valgrind_3.15.0.bb | 4 
  1 file changed, 4 insertions(+)


I did test whether we could get away with a dependency on valgrind-dbg
as in theory that should behave the same way. It doesn't which means
the debug symbol linkage isn't being honoured by valgrind and that is
something we need to look into too :/.


I tried that as well. When it didn't work and I couldn't find an
example that used that sort of dependency, I looked for other solutions
and ended up with the admittedly poor one here which only has the
one benefit that it works. ;-)

Ross asked about the disk usage impact:

du -sk for .../packages-split/foo

Type valgrindvalgrind-ptest
unstripped   169732  138192
stripped  27224   99836

ls -l /.../packages-split/usr/bin/valgrind
unstripped: 88672
  stripped: 22728

To be sure such bloat doesn't slip into the 2.8 release,
we should probably wait for the partial stripping ER implementation
and/or I can figure out what's wrong with adding a
   valgrind-ptest -> valgrind-dbg
dependency.



I'm torn on whether to accept this patch or try and fix partial
stripping...


Let's wait. See above.

We at least understand all but one of the valgrind ptest problems
that were happening on x86-64. The drd/tests/pth_detached3 [1] failure
seems to be with valgrind itself. The executable is meant to segfault
and it does:

root@qemux86-64:/usr/lib/valgrind/ptest# drd/tests/pth_detached3
[ 2148.508705] pth_detached3[9127]: segfault at 7ffc5acd0090 ip 
003a0460a15c sp 7ffc5accfa08 error 6 in 
libpthread-2.29.so[3a04607000+f000]
[ 2148.515650] Code: 10 00 00 00 f0 83 88 08 03 00 00 10 64 48 8b 3c 25 
00 03 00 00 e8 64 7c 00 00 0f 1f 40 00 8b 87 d0 02 00 00 85 c0 78 36 31 
c0 
Segmentation fault

but when run under valgrind, there's an extra frame in the stack trace:

root@qemux86-64:# cat drd/tests/pth_detached3.stderr.diff1
--- pth_detached3.stderr.exp1   2019-05-14 15:46:06.0 +
+++ pth_detached3.stderr.out2019-05-14 16:13:34.35500 +
@@ -1,12 +1,19 @@

 pthread_detach(): invalid thread ID 0x
-   at 0x: pthread_detach (drd_pthread_intercepts.c:?)
+   at 0x: vgDrd_set_joinable (drd_pthread_intercepts.c:?)
+   by 0x: pthread_detach (drd_pthread_intercepts.c:?)
by 0x: main (pth_detached3.c:21)
...

I need to learn a bit more before considering whether to just add
the missing lines to the expected output file to allow the test to pass.

qemuarm64 is still a problem that I intend to work on it soon.
ptests trigger the OOM killer even if I give the system 3GB of RAM
but I haven't really worked to understand why.

../Randy



Cheers,

Richard




--
# Randy MacLeod
# Wind River Linux


[1] drd/tests/pth_detached3.c

/* Invoke pthread_detach() with an invalid thread ID. */

#include 
#include 
#include 
#include 

static void* thread_func(void* arg)
{
  return 0;
}

int main(int argc, char** argv)
{
  pthread_t thread;

  pthread_create(&thread, NULL, thread_func, NULL);
  pthread_join(thread, NULL);

  /* Invoke pthread_detach() with the thread ID of a joined thread. */
  pthread_detach(thread);

  /* Invoke pthread_detach() with an invalid thread ID. */
  pthread_detach(thread + 8);

  fprintf(stderr, "Finished.\n");

  return 0;
}
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/5] valgrind: do not strip the package or ptests

2019-05-14 Thread richard . purdie 
On Tue, 2019-05-14 at 13:54 -0400, Randy MacLeod wrote:
> On 5/14/19 10:08 AM, Richard Purdie wrote:
> > On Mon, 2019-05-13 at 23:57 -0400, Randy MacLeod wrote:
> > > About half the ptests will fail if the executables deployed
> > > as part of the ptest package are stripped. Currently
> > > there is no easy way to only strip the main valgrind
> > > package so leave it and the ptests *all* unstripped.
> > > 
> > > There is an enhancement filed to enable wildcard stripping:
> > > https://bugzilla.yoctoproject.org/show_bug.cgi?id=13343
> > > so this recipe can be fixed once that is implemented.
> > > 
> > > Signed-off-by: Randy MacLeod 
> > > ---
> > >   meta/recipes-devtools/valgrind/valgrind_3.15.0.bb | 4 
> > >   1 file changed, 4 insertions(+)
> > 
> > I did test whether we could get away with a dependency on valgrind-
> > dbg
> > as in theory that should behave the same way. It doesn't which
> > means
> > the debug symbol linkage isn't being honoured by valgrind and that
> > is
> > something we need to look into too :/.
> 
> I tried that as well. When it didn't work and I couldn't find an
> example that used that sort of dependency, I looked for other
> solutions
> and ended up with the admittedly poor one here which only has the
> one benefit that it works. ;-)

Actually, my test was flawed. I just retested and this time it worked.
What I tested with was:

RDEPENDS_${PN}-ptest += " sed perl perl-module-file-glob ${PN} ${PN}-dbg"
INSANE_SKIP_${PN}-ptest = "debug-deps"

Can you see if that works for you?

We also have a problem that its overflowing the 4GB limit on some image
sizes with the debug info included :/

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/5] valgrind: do not strip the package or ptests

2019-05-14 Thread Randy MacLeod 

On 5/14/19 5:32 PM, richard.pur...@linuxfoundation.org wrote:

Actually, my test was flawed. I just retested and this time it worked.
What I tested with was:

RDEPENDS_${PN}-ptest += " sed perl perl-module-file-glob ${PN} ${PN}-dbg"
INSANE_SKIP_${PN}-ptest = "debug-deps"


Ah nice. Thanks.



Can you see if that works for you?


Sure. I'll test and resend the whole series.



We also have a problem that its overflowing the 4GB limit on some image
sizes with the debug info included :/


There's an enhancement to track that:
  https://bugzilla.yoctoproject.org/show_bug.cgi?id=13025
Kevin owns it and has it scheduled for 2.8 M2.

../Randy



Cheers,

Richard



--
# Randy MacLeod
# Wind River Linux
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/5] valgrind: do not strip the package or ptests

2019-05-14 Thread richard . purdie 
On Tue, 2019-05-14 at 17:42 -0400, Randy MacLeod wrote:
> On 5/14/19 5:32 PM, richard.pur...@linuxfoundation.org wrote:
> > Actually, my test was flawed. I just retested and this time it
> > worked.
> > What I tested with was:
> > 
> > RDEPENDS_${PN}-ptest += " sed perl perl-module-file-glob ${PN}
> > ${PN}-dbg"
> > INSANE_SKIP_${PN}-ptest = "debug-deps"
> 
> Ah nice. Thanks.

Note I add PN above, I'm going to send a different patch which does
that in the ptest bbclass as it fixes a ton of problems. Its not quite
a straightforward patch though.

> > Can you see if that works for you?
> 
> Sure. I'll test and resend the whole series.
> 
> > We also have a problem that its overflowing the 4GB limit on some
> > image
> > sizes with the debug info included :/
> 
> There's an enhancement to track that:
>https://bugzilla.yoctoproject.org/show_bug.cgi?id=13025
> Kevin owns it and has it scheduled for 2.8 M2.

The immediate problem will have to be fixed before valgrind can merge:

https://autobuilder.yoctoproject.org/typhoon/#/builders/37/builds/602

Cheers,

Richard

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/5] valgrind: do not strip the package or ptests

2019-05-14 Thread Mittal, Anuj 
On Tue, 2019-05-14 at 17:42 -0400, Randy MacLeod wrote:
> > We also have a problem that its overflowing the 4GB limit on some
> > image
> > sizes with the debug info included :/
> 
> There's an enhancement to track that:
>https://bugzilla.yoctoproject.org/show_bug.cgi?id=13025
> Kevin owns it and has it scheduled for 2.8 M2.

Chee Yang had sent patches that add 'install' feature to a wic image 
as a first step towards achieving this ... It'd be great if those can
be reviewed:

https://patchwork.openembedded.org/series/17478/

Thanks,

Anuj
-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/5] valgrind: do not strip the package or ptests

2019-05-14 Thread Randy MacLeod 

On 5/14/19 5:46 PM, richard.pur...@linuxfoundation.org wrote:

On Tue, 2019-05-14 at 17:42 -0400, Randy MacLeod wrote:

On 5/14/19 5:32 PM, richard.pur...@linuxfoundation.org wrote:

Actually, my test was flawed. I just retested and this time it
worked.
What I tested with was:

RDEPENDS_${PN}-ptest += " sed perl perl-module-file-glob ${PN}
${PN}-dbg"
INSANE_SKIP_${PN}-ptest = "debug-deps"


Ah nice. Thanks.


Note I add PN above, I'm going to send a different patch which does
that in the ptest bbclass as it fixes a ton of problems. Its not quite
a straightforward patch though.


Can you see if that works for you?


Sure. I'll test and resend the whole series.


We also have a problem that its overflowing the 4GB limit on some
image
sizes with the debug info included :/


There's an enhancement to track that:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=13025
Kevin owns it and has it scheduled for 2.8 M2.


The immediate problem will have to be fixed before valgrind can merge:

https://autobuilder.yoctoproject.org/typhoon/#/builders/37/builds/602

Cheers,

Richard



More test results!

The qemuarm64 valgrind ptest failures are due to a hard-coded
timeout patch in vg_regtest. It timed out at 30 seconds but
always left orphaned process hanging which eventually triggered
an OOM killer rampage. I bumped the timeout to 300 seconds to
see what that would do. It takes a long, long time but
the final result is the same as x86:

qemuarm64:
=== Test Summary ===
TOTAL: 159
PASSED: 149
FAILED: 1
SKIPPED: 9
DURATION: 10083
END: /usr/lib/valgrind/ptest
2019-05-14T23:16


qemuppc (300 second timeout):
=== Test Summary ===
TOTAL: 159
PASSED: 146
FAILED: 3
SKIPPED: 10
DURATION: 5492
END: /usr/lib/valgrind/ptest
2019-05-15T01:59
STOP: ptest-runner
root@qemuppc:/usr/lib/valgrind/ptest# grep FAIL 
valgrind_ptest_20190515-002816.log

FAIL: drd/tests/annotate_trace_memory
   but ^^^-- sorting exp-32bit and out file -> no diff
FAIL: drd/tests/annotate_trace_memory_xml <-- same
FAIL: drd/tests/std_list
FAILED: 3

I'm not sure why the output file is not ordered as expected but
I will likely not investigate qemuppc unless someone knows of a
use-case where this is an actual problem.


We need to agree on how to handle the ptest timeout.
I'll check if other recipes accept the optional timeout from:
   # ptest-runner -h
   Usage: ptest-runner [-d directory] [-e exclude] [-l list] \
 [-t timeout] [-x xml-filename] [-h] [ptest1 ptest2 ...]

It's tempting to just bump the timeout to 300 seconds
since anything that fails with that timeout really should
be fixed. The right thing is to pass the timeout from ptest-runner
down to valgrind's vg_regtest script. If we have people who set
timeouts to a non-default value and will be running valgrind's
ptest then I could do that.

Since I'm a completist, I'm going to try qemumips64 next! :)
--
# Randy MacLeod
# Wind River Linux
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/5] valgrind: do not strip the package or ptests

2019-05-15 Thread Randy MacLeod 

On 5/14/19 10:22 PM, Randy MacLeod wrote:

...



It's tempting to just bump the timeout to 300 seconds
since anything that fails with that timeout really should
be fixed. The right thing is to pass the timeout from ptest-runner
down to valgrind's vg_regtest script. If we have people who set
timeouts to a non-default value and will be running valgrind's
ptest then I could do that.

Since I'm a completist, I'm going to try qemumips64 next! :)


Okay, so maybe I'm not a MIPS64 completist!


qemumips64 hangs on drd/tests/annotate_barrier. I'm not sure why.

When run under gdb:

(gdb) bt
#0  vgPlain_machine_get_hwcaps () at 
../../valgrind-3.15.0/coregrind/m_machine.c:1910
#1  0x580b5a80 in valgrind_main (argc=, argv=0x580bb430 
<_start_in_C_linux+224>, envp=0x120015520)
at ../../valgrind-3.15.0/coregrind/m_main.c:1375
#2  0x580bb430 in _start_in_C_linux (pArgc=0xffabe0) at 
../../valgrind-3.15.0/coregrind/m_main.c:2985
#3  0x580b47e8 in __start ()
Backtrace stopped: frame did not save the PC
(gdb) bt full
#0  vgPlain_machine_get_hwcaps () at 
../../valgrind-3.15.0/coregrind/m_machine.c:1910
have_MSA = 1 '\001'
saved_set = {sig = {0, 0}}
tmp_set = {sig = {8, 0}}
saved_sigill_act = {sa_flags = 0, ksa_handler = 0x0, sa_mask = {sig = 
{0, 0}}, sa_restorer = 0xffae8e}
tmp_sigill_act = {sa_flags = 1073741824, ksa_handler = 0x58036f90 
, sa_mask = {sig = {0, 0}}, sa_restorer = 0xffae8e}
r = 
__PRETTY_FUNCTION__ = "vgPlain_machine_get_hwcaps"


1905  /* MSA instructions */
1906  have_MSA = True;
1907  if  (VG_MINIMAL_SETJMP(env_unsup_insn)) {
1908 have_MSA = False;
1909  }else  {
1910 __asm__ __volatile__(".word 0x7800088E");/* addv.b w2, w1, w0 
*/
1911  }

The qemumips64 ptests for valgrind are now on the back burner.

--
# Randy MacLeod
# Wind River Linux

-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/5] valgrind: do not strip the package or ptests

2019-05-20 Thread Khem Raj 




On 5/14/19 7:08 AM, Richard Purdie wrote:

On Mon, 2019-05-13 at 23:57 -0400, Randy MacLeod wrote:

About half the ptests will fail if the executables deployed
as part of the ptest package are stripped. Currently
there is no easy way to only strip the main valgrind
package so leave it and the ptests *all* unstripped.

There is an enhancement filed to enable wildcard stripping:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=13343
so this recipe can be fixed once that is implemented.

Signed-off-by: Randy MacLeod 
---
  meta/recipes-devtools/valgrind/valgrind_3.15.0.bb | 4 
  1 file changed, 4 insertions(+)


I did test whether we could get away with a dependency on valgrind-dbg
as in theory that should behave the same way. It doesn't which means
the debug symbol linkage isn't being honoured by valgrind and that is
something we need to look into too :/.

I'm torn on whether to accept this patch or try and fix partial
stripping...


we need PN-syms and PN-dbg and then we can add rdep on PN-syms alone
but sadly thats another hump to climb.



Cheers,

Richard


--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [OE-core] [PATCH 2/5] valgrind: do not strip the package or ptests

2019-05-20 Thread Khem Raj 




On 5/15/19 12:11 PM, Randy MacLeod wrote:

On 5/14/19 10:22 PM, Randy MacLeod wrote:

...



It's tempting to just bump the timeout to 300 seconds
since anything that fails with that timeout really should
be fixed. The right thing is to pass the timeout from ptest-runner
down to valgrind's vg_regtest script. If we have people who set
timeouts to a non-default value and will be running valgrind's
ptest then I could do that.

Since I'm a completist, I'm going to try qemumips64 next! :)


Okay, so maybe I'm not a MIPS64 completist!




Let me try to help a bit.


qemumips64 hangs on drd/tests/annotate_barrier. I'm not sure why.

When run under gdb:

(gdb) bt
#0  vgPlain_machine_get_hwcaps () at 
../../valgrind-3.15.0/coregrind/m_machine.c:1910
#1  0x580b5a80 in valgrind_main (argc=, argv=0x580bb430 
<_start_in_C_linux+224>, envp=0x120015520)
 at ../../valgrind-3.15.0/coregrind/m_main.c:1375
#2  0x580bb430 in _start_in_C_linux (pArgc=0xffabe0) at 
../../valgrind-3.15.0/coregrind/m_main.c:2985
#3  0x580b47e8 in __start ()
Backtrace stopped: frame did not save the PC
(gdb) bt full
#0  vgPlain_machine_get_hwcaps () at 
../../valgrind-3.15.0/coregrind/m_machine.c:1910
 have_MSA = 1 '\001'
 saved_set = {sig = {0, 0}}
 tmp_set = {sig = {8, 0}}
 saved_sigill_act = {sa_flags = 0, ksa_handler = 0x0, sa_mask = {sig = 
{0, 0}}, sa_restorer = 0xffae8e}
 tmp_sigill_act = {sa_flags = 1073741824, ksa_handler = 0x58036f90 
, sa_mask = {sig = {0, 0}}, sa_restorer = 0xffae8e}
 r = 
 __PRETTY_FUNCTION__ = "vgPlain_machine_get_hwcaps"


1905  /* MSA instructions */
1906  have_MSA = True;
1907  if  (VG_MINIMAL_SETJMP(env_unsup_insn)) {
1908 have_MSA = False;
1909  }else  {
1910 __asm__ __volatile__(".word 0x7800088E");/* addv.b w2, w1, w0 
*/



I think its stuck executing this instruction which is checking for ASE 
support and I guess we do not have ASE enabled in qemumips64 base arch 
emulation, or the instruction behaves differently on 32bit and 64bit ABIs




1911  }

The qemumips64 ptests for valgrind are now on the back burner.





--
# Randy MacLeod
# Wind River Linux



--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core