[OE-core] [PATCH 2/5] valgrind: do not strip the package or ptests
About half the ptests will fail if the executables deployed as part of the ptest package are stripped. Currently there is no easy way to only strip the main valgrind package so leave it and the ptests *all* unstripped. There is an enhancement filed to enable wildcard stripping: https://bugzilla.yoctoproject.org/show_bug.cgi?id=13343 so this recipe can be fixed once that is implemented. Signed-off-by: Randy MacLeod --- meta/recipes-devtools/valgrind/valgrind_3.15.0.bb | 4 1 file changed, 4 insertions(+) diff --git a/meta/recipes-devtools/valgrind/valgrind_3.15.0.bb b/meta/recipes-devtools/valgrind/valgrind_3.15.0.bb index 9749fdb244..b72f4d76ee 100644 --- a/meta/recipes-devtools/valgrind/valgrind_3.15.0.bb +++ b/meta/recipes-devtools/valgrind/valgrind_3.15.0.bb @@ -114,6 +114,10 @@ RDEPENDS_${PN}-ptest_append_libc-glibc = " glibc-utils" # One of the tests contains a bogus interpreter path on purpose. # Skip file dependency check SKIP_FILEDEPS_${PN}-ptest = '1' +# INHIBIT_PACKAGE_STRIP_${PN}-ptest = '1' +# PACKAGE_DEBUG_SPLIT_STYLE_${PN}-ptest = 'debug-file-directory' +INHIBIT_PACKAGE_STRIP = '1' +PACKAGE_DEBUG_SPLIT_STYLE = 'debug-file-directory' do_compile_ptest() { oe_runmake check -- 2.17.0 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 2/5] valgrind: do not strip the package or ptests
On Mon, 2019-05-13 at 23:57 -0400, Randy MacLeod wrote: > About half the ptests will fail if the executables deployed > as part of the ptest package are stripped. Currently > there is no easy way to only strip the main valgrind > package so leave it and the ptests *all* unstripped. > > There is an enhancement filed to enable wildcard stripping: >https://bugzilla.yoctoproject.org/show_bug.cgi?id=13343 > so this recipe can be fixed once that is implemented. > > Signed-off-by: Randy MacLeod > --- > meta/recipes-devtools/valgrind/valgrind_3.15.0.bb | 4 > 1 file changed, 4 insertions(+) I did test whether we could get away with a dependency on valgrind-dbg as in theory that should behave the same way. It doesn't which means the debug symbol linkage isn't being honoured by valgrind and that is something we need to look into too :/. I'm torn on whether to accept this patch or try and fix partial stripping... Cheers, Richard -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 2/5] valgrind: do not strip the package or ptests
On Mon, 13 May 2019 at 20:57, Randy MacLeod wrote: > About half the ptests will fail if the executables deployed > as part of the ptest package are stripped. Currently > there is no easy way to only strip the main valgrind > package so leave it and the ptests *all* unstripped. What's the disk usage impact of an unstripped valgrind package? Ross -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 2/5] valgrind: do not strip the package or ptests
On 5/14/19 10:08 AM, Richard Purdie wrote: On Mon, 2019-05-13 at 23:57 -0400, Randy MacLeod wrote: About half the ptests will fail if the executables deployed as part of the ptest package are stripped. Currently there is no easy way to only strip the main valgrind package so leave it and the ptests *all* unstripped. There is an enhancement filed to enable wildcard stripping: https://bugzilla.yoctoproject.org/show_bug.cgi?id=13343 so this recipe can be fixed once that is implemented. Signed-off-by: Randy MacLeod --- meta/recipes-devtools/valgrind/valgrind_3.15.0.bb | 4 1 file changed, 4 insertions(+) I did test whether we could get away with a dependency on valgrind-dbg as in theory that should behave the same way. It doesn't which means the debug symbol linkage isn't being honoured by valgrind and that is something we need to look into too :/. I tried that as well. When it didn't work and I couldn't find an example that used that sort of dependency, I looked for other solutions and ended up with the admittedly poor one here which only has the one benefit that it works. ;-) Ross asked about the disk usage impact: du -sk for .../packages-split/foo Type valgrindvalgrind-ptest unstripped 169732 138192 stripped 27224 99836 ls -l /.../packages-split/usr/bin/valgrind unstripped: 88672 stripped: 22728 To be sure such bloat doesn't slip into the 2.8 release, we should probably wait for the partial stripping ER implementation and/or I can figure out what's wrong with adding a valgrind-ptest -> valgrind-dbg dependency. I'm torn on whether to accept this patch or try and fix partial stripping... Let's wait. See above. We at least understand all but one of the valgrind ptest problems that were happening on x86-64. The drd/tests/pth_detached3 [1] failure seems to be with valgrind itself. The executable is meant to segfault and it does: root@qemux86-64:/usr/lib/valgrind/ptest# drd/tests/pth_detached3 [ 2148.508705] pth_detached3[9127]: segfault at 7ffc5acd0090 ip 003a0460a15c sp 7ffc5accfa08 error 6 in libpthread-2.29.so[3a04607000+f000] [ 2148.515650] Code: 10 00 00 00 f0 83 88 08 03 00 00 10 64 48 8b 3c 25 00 03 00 00 e8 64 7c 00 00 0f 1f 40 00 8b 87 d0 02 00 00 85 c0 78 36 31 c0 Segmentation fault but when run under valgrind, there's an extra frame in the stack trace: root@qemux86-64:# cat drd/tests/pth_detached3.stderr.diff1 --- pth_detached3.stderr.exp1 2019-05-14 15:46:06.0 + +++ pth_detached3.stderr.out2019-05-14 16:13:34.35500 + @@ -1,12 +1,19 @@ pthread_detach(): invalid thread ID 0x - at 0x: pthread_detach (drd_pthread_intercepts.c:?) + at 0x: vgDrd_set_joinable (drd_pthread_intercepts.c:?) + by 0x: pthread_detach (drd_pthread_intercepts.c:?) by 0x: main (pth_detached3.c:21) ... I need to learn a bit more before considering whether to just add the missing lines to the expected output file to allow the test to pass. qemuarm64 is still a problem that I intend to work on it soon. ptests trigger the OOM killer even if I give the system 3GB of RAM but I haven't really worked to understand why. ../Randy Cheers, Richard -- # Randy MacLeod # Wind River Linux [1] drd/tests/pth_detached3.c /* Invoke pthread_detach() with an invalid thread ID. */ #include #include #include #include static void* thread_func(void* arg) { return 0; } int main(int argc, char** argv) { pthread_t thread; pthread_create(&thread, NULL, thread_func, NULL); pthread_join(thread, NULL); /* Invoke pthread_detach() with the thread ID of a joined thread. */ pthread_detach(thread); /* Invoke pthread_detach() with an invalid thread ID. */ pthread_detach(thread + 8); fprintf(stderr, "Finished.\n"); return 0; } -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 2/5] valgrind: do not strip the package or ptests
On Tue, 2019-05-14 at 13:54 -0400, Randy MacLeod wrote: > On 5/14/19 10:08 AM, Richard Purdie wrote: > > On Mon, 2019-05-13 at 23:57 -0400, Randy MacLeod wrote: > > > About half the ptests will fail if the executables deployed > > > as part of the ptest package are stripped. Currently > > > there is no easy way to only strip the main valgrind > > > package so leave it and the ptests *all* unstripped. > > > > > > There is an enhancement filed to enable wildcard stripping: > > > https://bugzilla.yoctoproject.org/show_bug.cgi?id=13343 > > > so this recipe can be fixed once that is implemented. > > > > > > Signed-off-by: Randy MacLeod > > > --- > > > meta/recipes-devtools/valgrind/valgrind_3.15.0.bb | 4 > > > 1 file changed, 4 insertions(+) > > > > I did test whether we could get away with a dependency on valgrind- > > dbg > > as in theory that should behave the same way. It doesn't which > > means > > the debug symbol linkage isn't being honoured by valgrind and that > > is > > something we need to look into too :/. > > I tried that as well. When it didn't work and I couldn't find an > example that used that sort of dependency, I looked for other > solutions > and ended up with the admittedly poor one here which only has the > one benefit that it works. ;-) Actually, my test was flawed. I just retested and this time it worked. What I tested with was: RDEPENDS_${PN}-ptest += " sed perl perl-module-file-glob ${PN} ${PN}-dbg" INSANE_SKIP_${PN}-ptest = "debug-deps" Can you see if that works for you? We also have a problem that its overflowing the 4GB limit on some image sizes with the debug info included :/ Cheers, Richard -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 2/5] valgrind: do not strip the package or ptests
On 5/14/19 5:32 PM, richard.pur...@linuxfoundation.org wrote: Actually, my test was flawed. I just retested and this time it worked. What I tested with was: RDEPENDS_${PN}-ptest += " sed perl perl-module-file-glob ${PN} ${PN}-dbg" INSANE_SKIP_${PN}-ptest = "debug-deps" Ah nice. Thanks. Can you see if that works for you? Sure. I'll test and resend the whole series. We also have a problem that its overflowing the 4GB limit on some image sizes with the debug info included :/ There's an enhancement to track that: https://bugzilla.yoctoproject.org/show_bug.cgi?id=13025 Kevin owns it and has it scheduled for 2.8 M2. ../Randy Cheers, Richard -- # Randy MacLeod # Wind River Linux -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 2/5] valgrind: do not strip the package or ptests
On Tue, 2019-05-14 at 17:42 -0400, Randy MacLeod wrote: > On 5/14/19 5:32 PM, richard.pur...@linuxfoundation.org wrote: > > Actually, my test was flawed. I just retested and this time it > > worked. > > What I tested with was: > > > > RDEPENDS_${PN}-ptest += " sed perl perl-module-file-glob ${PN} > > ${PN}-dbg" > > INSANE_SKIP_${PN}-ptest = "debug-deps" > > Ah nice. Thanks. Note I add PN above, I'm going to send a different patch which does that in the ptest bbclass as it fixes a ton of problems. Its not quite a straightforward patch though. > > Can you see if that works for you? > > Sure. I'll test and resend the whole series. > > > We also have a problem that its overflowing the 4GB limit on some > > image > > sizes with the debug info included :/ > > There's an enhancement to track that: >https://bugzilla.yoctoproject.org/show_bug.cgi?id=13025 > Kevin owns it and has it scheduled for 2.8 M2. The immediate problem will have to be fixed before valgrind can merge: https://autobuilder.yoctoproject.org/typhoon/#/builders/37/builds/602 Cheers, Richard -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 2/5] valgrind: do not strip the package or ptests
On Tue, 2019-05-14 at 17:42 -0400, Randy MacLeod wrote: > > We also have a problem that its overflowing the 4GB limit on some > > image > > sizes with the debug info included :/ > > There's an enhancement to track that: >https://bugzilla.yoctoproject.org/show_bug.cgi?id=13025 > Kevin owns it and has it scheduled for 2.8 M2. Chee Yang had sent patches that add 'install' feature to a wic image as a first step towards achieving this ... It'd be great if those can be reviewed: https://patchwork.openembedded.org/series/17478/ Thanks, Anuj -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 2/5] valgrind: do not strip the package or ptests
On 5/14/19 5:46 PM, richard.pur...@linuxfoundation.org wrote: On Tue, 2019-05-14 at 17:42 -0400, Randy MacLeod wrote: On 5/14/19 5:32 PM, richard.pur...@linuxfoundation.org wrote: Actually, my test was flawed. I just retested and this time it worked. What I tested with was: RDEPENDS_${PN}-ptest += " sed perl perl-module-file-glob ${PN} ${PN}-dbg" INSANE_SKIP_${PN}-ptest = "debug-deps" Ah nice. Thanks. Note I add PN above, I'm going to send a different patch which does that in the ptest bbclass as it fixes a ton of problems. Its not quite a straightforward patch though. Can you see if that works for you? Sure. I'll test and resend the whole series. We also have a problem that its overflowing the 4GB limit on some image sizes with the debug info included :/ There's an enhancement to track that: https://bugzilla.yoctoproject.org/show_bug.cgi?id=13025 Kevin owns it and has it scheduled for 2.8 M2. The immediate problem will have to be fixed before valgrind can merge: https://autobuilder.yoctoproject.org/typhoon/#/builders/37/builds/602 Cheers, Richard More test results! The qemuarm64 valgrind ptest failures are due to a hard-coded timeout patch in vg_regtest. It timed out at 30 seconds but always left orphaned process hanging which eventually triggered an OOM killer rampage. I bumped the timeout to 300 seconds to see what that would do. It takes a long, long time but the final result is the same as x86: qemuarm64: === Test Summary === TOTAL: 159 PASSED: 149 FAILED: 1 SKIPPED: 9 DURATION: 10083 END: /usr/lib/valgrind/ptest 2019-05-14T23:16 qemuppc (300 second timeout): === Test Summary === TOTAL: 159 PASSED: 146 FAILED: 3 SKIPPED: 10 DURATION: 5492 END: /usr/lib/valgrind/ptest 2019-05-15T01:59 STOP: ptest-runner root@qemuppc:/usr/lib/valgrind/ptest# grep FAIL valgrind_ptest_20190515-002816.log FAIL: drd/tests/annotate_trace_memory but ^^^-- sorting exp-32bit and out file -> no diff FAIL: drd/tests/annotate_trace_memory_xml <-- same FAIL: drd/tests/std_list FAILED: 3 I'm not sure why the output file is not ordered as expected but I will likely not investigate qemuppc unless someone knows of a use-case where this is an actual problem. We need to agree on how to handle the ptest timeout. I'll check if other recipes accept the optional timeout from: # ptest-runner -h Usage: ptest-runner [-d directory] [-e exclude] [-l list] \ [-t timeout] [-x xml-filename] [-h] [ptest1 ptest2 ...] It's tempting to just bump the timeout to 300 seconds since anything that fails with that timeout really should be fixed. The right thing is to pass the timeout from ptest-runner down to valgrind's vg_regtest script. If we have people who set timeouts to a non-default value and will be running valgrind's ptest then I could do that. Since I'm a completist, I'm going to try qemumips64 next! :) -- # Randy MacLeod # Wind River Linux -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 2/5] valgrind: do not strip the package or ptests
On 5/14/19 10:22 PM, Randy MacLeod wrote: ... It's tempting to just bump the timeout to 300 seconds since anything that fails with that timeout really should be fixed. The right thing is to pass the timeout from ptest-runner down to valgrind's vg_regtest script. If we have people who set timeouts to a non-default value and will be running valgrind's ptest then I could do that. Since I'm a completist, I'm going to try qemumips64 next! :) Okay, so maybe I'm not a MIPS64 completist! qemumips64 hangs on drd/tests/annotate_barrier. I'm not sure why. When run under gdb: (gdb) bt #0 vgPlain_machine_get_hwcaps () at ../../valgrind-3.15.0/coregrind/m_machine.c:1910 #1 0x580b5a80 in valgrind_main (argc=, argv=0x580bb430 <_start_in_C_linux+224>, envp=0x120015520) at ../../valgrind-3.15.0/coregrind/m_main.c:1375 #2 0x580bb430 in _start_in_C_linux (pArgc=0xffabe0) at ../../valgrind-3.15.0/coregrind/m_main.c:2985 #3 0x580b47e8 in __start () Backtrace stopped: frame did not save the PC (gdb) bt full #0 vgPlain_machine_get_hwcaps () at ../../valgrind-3.15.0/coregrind/m_machine.c:1910 have_MSA = 1 '\001' saved_set = {sig = {0, 0}} tmp_set = {sig = {8, 0}} saved_sigill_act = {sa_flags = 0, ksa_handler = 0x0, sa_mask = {sig = {0, 0}}, sa_restorer = 0xffae8e} tmp_sigill_act = {sa_flags = 1073741824, ksa_handler = 0x58036f90 , sa_mask = {sig = {0, 0}}, sa_restorer = 0xffae8e} r = __PRETTY_FUNCTION__ = "vgPlain_machine_get_hwcaps" 1905 /* MSA instructions */ 1906 have_MSA = True; 1907 if (VG_MINIMAL_SETJMP(env_unsup_insn)) { 1908 have_MSA = False; 1909 }else { 1910 __asm__ __volatile__(".word 0x7800088E");/* addv.b w2, w1, w0 */ 1911 } The qemumips64 ptests for valgrind are now on the back burner. -- # Randy MacLeod # Wind River Linux -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 2/5] valgrind: do not strip the package or ptests
On 5/14/19 7:08 AM, Richard Purdie wrote: On Mon, 2019-05-13 at 23:57 -0400, Randy MacLeod wrote: About half the ptests will fail if the executables deployed as part of the ptest package are stripped. Currently there is no easy way to only strip the main valgrind package so leave it and the ptests *all* unstripped. There is an enhancement filed to enable wildcard stripping: https://bugzilla.yoctoproject.org/show_bug.cgi?id=13343 so this recipe can be fixed once that is implemented. Signed-off-by: Randy MacLeod --- meta/recipes-devtools/valgrind/valgrind_3.15.0.bb | 4 1 file changed, 4 insertions(+) I did test whether we could get away with a dependency on valgrind-dbg as in theory that should behave the same way. It doesn't which means the debug symbol linkage isn't being honoured by valgrind and that is something we need to look into too :/. I'm torn on whether to accept this patch or try and fix partial stripping... we need PN-syms and PN-dbg and then we can add rdep on PN-syms alone but sadly thats another hump to climb. Cheers, Richard -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 2/5] valgrind: do not strip the package or ptests
On 5/15/19 12:11 PM, Randy MacLeod wrote: On 5/14/19 10:22 PM, Randy MacLeod wrote: ... It's tempting to just bump the timeout to 300 seconds since anything that fails with that timeout really should be fixed. The right thing is to pass the timeout from ptest-runner down to valgrind's vg_regtest script. If we have people who set timeouts to a non-default value and will be running valgrind's ptest then I could do that. Since I'm a completist, I'm going to try qemumips64 next! :) Okay, so maybe I'm not a MIPS64 completist! Let me try to help a bit. qemumips64 hangs on drd/tests/annotate_barrier. I'm not sure why. When run under gdb: (gdb) bt #0 vgPlain_machine_get_hwcaps () at ../../valgrind-3.15.0/coregrind/m_machine.c:1910 #1 0x580b5a80 in valgrind_main (argc=, argv=0x580bb430 <_start_in_C_linux+224>, envp=0x120015520) at ../../valgrind-3.15.0/coregrind/m_main.c:1375 #2 0x580bb430 in _start_in_C_linux (pArgc=0xffabe0) at ../../valgrind-3.15.0/coregrind/m_main.c:2985 #3 0x580b47e8 in __start () Backtrace stopped: frame did not save the PC (gdb) bt full #0 vgPlain_machine_get_hwcaps () at ../../valgrind-3.15.0/coregrind/m_machine.c:1910 have_MSA = 1 '\001' saved_set = {sig = {0, 0}} tmp_set = {sig = {8, 0}} saved_sigill_act = {sa_flags = 0, ksa_handler = 0x0, sa_mask = {sig = {0, 0}}, sa_restorer = 0xffae8e} tmp_sigill_act = {sa_flags = 1073741824, ksa_handler = 0x58036f90 , sa_mask = {sig = {0, 0}}, sa_restorer = 0xffae8e} r = __PRETTY_FUNCTION__ = "vgPlain_machine_get_hwcaps" 1905 /* MSA instructions */ 1906 have_MSA = True; 1907 if (VG_MINIMAL_SETJMP(env_unsup_insn)) { 1908 have_MSA = False; 1909 }else { 1910 __asm__ __volatile__(".word 0x7800088E");/* addv.b w2, w1, w0 */ I think its stuck executing this instruction which is checking for ASE support and I guess we do not have ASE enabled in qemumips64 base arch emulation, or the instruction behaves differently on 32bit and 64bit ABIs 1911 } The qemumips64 ptests for valgrind are now on the back burner. -- # Randy MacLeod # Wind River Linux -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core