Bug#775744: libapp-stacktrace-perl: FTBFS: Failed 1/4 test programs. 1/6 subtests failed.

[Axel Beckert]

Hi Niko,

Niko Tyni wrote:
> > Actually it FTBFS on amd64, too, but only inside pbuilder. That smells
> > like a missing build-dependency to me. Will dig into that direction
> > for the moment…
> 
> It needs perl-debug. Looks like the return value of Perl_get_context()
> is taken to be an int if there is no symbol table indicating otherwise.
> 
> Probably perl-debug should be both a build and a runtime dependency.

Thanks for that hint. I was looking into libc6-dev and libperl-dev but
didn't think about perl-debug. Will check and upload once I got it
working.

Regards, Axel
-- 
 ,''`.  |  Axel Beckert , http://people.debian.org/~abe/
: :' :  |  Debian Developer, ftp.ch.debian.org Admin
`. `'   |  4096R: 2517 B724 C5F6 CA99 5329  6E61 2FF9 CD59 6126 16B5
  `-|  1024D: F067 EA27 26B9 C3FC 1486  202E C09E 1D89 9593 0EDE

Bug#775744: libapp-stacktrace-perl: FTBFS: Failed 1/4 test programs. 1/6 subtests failed.

[Niko Tyni]

On Sat, Dec 10, 2016 at 03:58:56AM +0100, Axel Beckert wrote:
> Axel Beckert wrote:
> > Axel Beckert wrote:
> > > Niko Tyni wrote:
> > > > The attached patch makes the test suite pass again for me on amd64.
> > > 
> > > Thanks. The patch works fine for me, too, at least on amd64 and armhf.
> > > But it now FTBFS on arm64 (where it built before):
> 
> Actually it FTBFS on amd64, too, but only inside pbuilder. That smells
> like a missing build-dependency to me. Will dig into that direction
> for the moment…

It needs perl-debug. Looks like the return value of Perl_get_context()
is taken to be an int if there is no symbol table indicating otherwise.

Probably perl-debug should be both a build and a runtime dependency.
-- 
Niko

Bug#775744: libapp-stacktrace-perl: FTBFS: Failed 1/4 test programs. 1/6 subtests failed.

[Axel Beckert]

Control: tag -1 - pending

Hi again,

Axel Beckert wrote:
> Axel Beckert wrote:
> > Niko Tyni wrote:
> > > The attached patch makes the test suite pass again for me on amd64.
> > 
> > Thanks. The patch works fine for me, too, at least on amd64 and armhf.
> > But it now FTBFS on arm64 (where it built before):

Actually it FTBFS on amd64, too, but only inside pbuilder. That smells
like a missing build-dependency to me. Will dig into that direction
for the moment…

Regards, Axel
-- 
 ,''`.  |  Axel Beckert , http://people.debian.org/~abe/
: :' :  |  Debian Developer, ftp.ch.debian.org Admin
`. `'   |  4096R: 2517 B724 C5F6 CA99 5329  6E61 2FF9 CD59 6126 16B5
  `-|  1024D: F067 EA27 26B9 C3FC 1486  202E C09E 1D89 9593 0EDE

Bug#775744: libapp-stacktrace-perl: FTBFS: Failed 1/4 test programs. 1/6 subtests failed.

[Axel Beckert]

Hi,

Axel Beckert wrote:
> Niko Tyni wrote:
> > The attached patch makes the test suite pass again for me on amd64.
> 
> Thanks. The patch works fine for me, too, at least on amd64 and armhf.
> But it now FTBFS on arm64 (where it built before):
> 
> […]
> t/release-pod-syntax.t  skipped: these tests are for release candidate 
> testing
> # [Thread debugging using libthread_db enabled]
> # Using host libthread_db library "/lib/aarch64-linux-gnu/libthread_db.so.1".
> # 0x007f993c2448 in select () from /lib/aarch64-linux-gnu/libc.so.6
> # /tmp/iDQWSbZWdw.gdb:1761: Error in sourced command file:
> # Cannot access memory at address 0x8d7da768
> 
> #   Failed test at t/unthreaded.t line 55.
> #   '[Thread debugging using libthread_db enabled]
> # Using host libthread_db library "/lib/aarch64-linux-gnu/libthread_db.so.1".
> # 0x007f993c2448 in select () from /lib/aarch64-linux-gnu/libc.so.6
> # /tmp/iDQWSbZWdw.gdb:1761: Error in sourced command file:
> # Cannot access memory at address 0x8d7da768

Meh, I pasted the wrong output. That was without the patch. With the
patch it still FTBFS on arm64, though:

# [Thread debugging using libthread_db enabled]
# Using host libthread_db library "/lib/aarch64-linux-gnu/libthread_db.so.1".
# 0x007fadfd2448 in select () from /lib/aarch64-linux-gnu/libc.so.6
# /tmp/S0cICGJg6x.gdb:1761: Error in sourced command file:
# Cannot access memory at address 0x59702768

#   Failed test at t/unthreaded.t line 55.
#   '[Thread debugging using libthread_db enabled]
# Using host libthread_db library "/lib/aarch64-linux-gnu/libthread_db.so.1".
# 0x007fadfd2448 in select () from /lib/aarch64-linux-gnu/libc.so.6
# /tmp/S0cICGJg6x.gdb:1761: Error in sourced command file:
# Cannot access memory at address 0x59702768
[…]
$ quilt applied
debian/patches/0001-Pointers-should-be-long-not-int.patch


Regards, Axel
-- 
 ,''`.  |  Axel Beckert , http://people.debian.org/~abe/
: :' :  |  Debian Developer, ftp.ch.debian.org Admin
`. `'   |  4096R: 2517 B724 C5F6 CA99 5329  6E61 2FF9 CD59 6126 16B5
  `-|  1024D: F067 EA27 26B9 C3FC 1486  202E C09E 1D89 9593 0EDE

Bug#775744: libapp-stacktrace-perl: FTBFS: Failed 1/4 test programs. 1/6 subtests failed.

[Axel Beckert]

Control: tag -1 + pending

Hi Niko,

Niko Tyni wrote:
> The attached patch makes the test suite pass again for me on amd64.

Thanks. The patch works fine for me, too, at least on amd64 and armhf.
But it now FTBFS on arm64 (where it built before):

[…]
t/release-pod-syntax.t  skipped: these tests are for release candidate 
testing
# [Thread debugging using libthread_db enabled]
# Using host libthread_db library "/lib/aarch64-linux-gnu/libthread_db.so.1".
# 0x007f993c2448 in select () from /lib/aarch64-linux-gnu/libc.so.6
# /tmp/iDQWSbZWdw.gdb:1761: Error in sourced command file:
# Cannot access memory at address 0x8d7da768

#   Failed test at t/unthreaded.t line 55.
#   '[Thread debugging using libthread_db enabled]
# Using host libthread_db library "/lib/aarch64-linux-gnu/libthread_db.so.1".
# 0x007f993c2448 in select () from /lib/aarch64-linux-gnu/libc.so.6
# /tmp/iDQWSbZWdw.gdb:1761: Error in sourced command file:
# Cannot access memory at address 0x8d7da768
# '
# doesn't match '(?^mx:
# (?:
# ^t/unthreaded\.t:\d+\n
# ){10}
# )'
# Looks like you failed 1 test of 5.
t/unthreaded.t  
1..5
not ok 1
ok 2 - Reaped pstack
ok 3 - exit(0)
ok 4 - No signals
ok 5 - No core dump
# Alive top 1481337347: 1 middle: 1
# Alive top 1481337348: 1 middle: 0
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/5 subtests 

Test Summary Report
---
t/unthreaded.t  (Wstat: 256 Tests: 5 Failed: 1)
  Failed test:  1
  Non-zero exit status: 1
Files=4, Tests=6,  2 wallclock secs ( 0.09 usr  0.02 sys +  1.35 cusr  0.12 
csys =  1.58 CPU)
Result: FAIL
Failed 1/4 test programs. 1/6 subtests failed.
Makefile:1009: recipe for target 'test_dynamic' failed
make[1]: *** [test_dynamic] Error 1
make[1]: Leaving directory '/home/abe/libapp-stacktrace-perl'
dh_auto_test: make -j1 test TEST_VERBOSE=1 returned exit code 2
[…]

Regards, Axel
-- 
 ,''`.  |  Axel Beckert , http://people.debian.org/~abe/
: :' :  |  Debian Developer, ftp.ch.debian.org Admin
`. `'   |  4096R: 2517 B724 C5F6 CA99 5329  6E61 2FF9 CD59 6126 16B5
  `-|  1024D: F067 EA27 26B9 C3FC 1486  202E C09E 1D89 9593 0EDE

Bug#775744: libapp-stacktrace-perl: FTBFS: Failed 1/4 test programs. 1/6 subtests failed.

[Niko Tyni]

Control: tag -1 patch

On Sun, Dec 04, 2016 at 05:35:22PM +0200, Niko Tyni wrote:

> The problem itself seems to be that the gdb macros in
>  lib/App/Stacktrace/perl_backtrace_raw.txt
> cast pointers to the C 'int' type, which is broken on 64-bit
> architectures. For instance, in 'perl_backtrace_5_12_x':
> set $interpreter = (int) Perl_get_context()
> 
> I suppose it used to work on amd64 because perl used low memory addresses
> so things didn't overflow, but that has changed with the latest perl
> upload, presumably due to gcc PIE changes.

The attached patch makes the test suite pass again for me on amd64.

It should probably be tested on other architectures but one way
to achieve that is to just upload it to experimental or sid.

As noted in the commit message, the thread dumping parts are untested,
and I tried not to touch other code paths than the one we're currently
taking. I might well have got something wrong though; the test suite
is unfortunately not very extensive.

I don't see any rdeps in the archive. Axel: if you are actually using
this module for something, testing there would be very welcome.

Upstream should of course be informed too, though they don't seem
very active (last release was in 2011.)
-- 
Niko Tyni   nt...@debian.org
>From 6712a7b680bf5e88cf30a5251b90872418c320f2 Mon Sep 17 00:00:00 2001
From: Niko Tyni 
Date: Tue, 6 Dec 2016 13:56:46 +0200
Subject: [PATCH] Pointers should be long not int

Casting pointers to int types breaks on 64-bit platforms where Perl uses
memory areas with pointers over 32 bits, which seems to be the case for
PIE (position independent executable) builds.

This only touches the code path relevant to Debian; 5.8, 5.10,
and unthreaded code paths are intended to be unaffected.

The changes to thread dumping parts are untested as they are not covered
by the test suite.

Bug-Debian: https://bugs.debian.org/775744
---
 lib/App/Stacktrace/perl_backtrace_raw.txt | 118 +++---
 1 file changed, 59 insertions(+), 59 deletions(-)

diff --git a/lib/App/Stacktrace/perl_backtrace_raw.txt b/lib/App/Stacktrace/perl_backtrace_raw.txt
index 9480e4d..41cf4e6 100644
--- a/lib/App/Stacktrace/perl_backtrace_raw.txt
+++ b/lib/App/Stacktrace/perl_backtrace_raw.txt
@@ -60,13 +60,13 @@ define perl_backtrace_an_interp
 printf "interpreter=%#x\n", $interpreter
 if $interpreter
 x/128xw $interpreter
-print $stackinfo = (int) *((int*) ($INTERPRETER_curstackinfo + (int) $interpreter))
+print $stackinfo = (long) *((long*) ($INTERPRETER_curstackinfo + (long) $interpreter))
 else
-print $stackinfo = (int) *((int*) ($INTERPRETER_curstackinfo + (int) $interpreter))
+print $stackinfo = (long) *((long*) ($INTERPRETER_curstackinfo + (long) $interpreter))
 end
 else
 if $interpreter
-set $stackinfo = (int) *((int*) ($INTERPRETER_curstackinfo + (int) $interpreter))
+set $stackinfo = (long) *((long*) ($INTERPRETER_curstackinfo + (long) $interpreter))
 else
 set $stackinfo = 0
 end
@@ -76,13 +76,13 @@ define perl_backtrace_an_interp
 printf "stackinfo=%#x\n", $stackinfo
 if $stackinfo
 x/128xw $stackinfo
-print $cxstack = (int) *((int*) ($STACKINFO_cxstack + (int) $stackinfo))
+print $cxstack = (long) *((long*) ($STACKINFO_cxstack + (long) $stackinfo))
 else
-print $cxstack = (int) *((int*) ($STACKINFO_cxstack + (int) $stackinfo))
+print $cxstack = (long) *((long*) ($STACKINFO_cxstack + (long) $stackinfo))
 end
 else
 if $stackinfo
-set $cxstack = (int) *((int*) ($STACKINFO_cxstack + (int) $stackinfo))
+set $cxstack = (long) *((long*) ($STACKINFO_cxstack + (long) $stackinfo))
 else
 set $cxstack = 0
 end
@@ -91,13 +91,13 @@ define perl_backtrace_an_interp
 printf "stackinfo=%#x\n", $stackinfo
 if $stackinfo
 x/128xw $stackinfo
-print $cxix = (int) *((int*) ($STACKINFO_cxix + (int) $stackinfo))
+print $cxix = (int) *((int*) ($STACKINFO_cxix + (long) $stackinfo))
 else
-print $cxix = (int) *((int*) ($STACKINFO_cxix + (int) $stackinfo))
+print $cxix = (int) *((int*) ($STACKINFO_cxix + (long) $stackinfo))
 end
 else
 if $stackinfo
-set $cxix = (int) *((int*) ($STACKINFO_cxix + (int) $stackinfo))
+set $cxix = (int) *((int*) ($STACKINFO_cxix + (long) $stackinfo))
 else
 set $cxix = 0
 end
@@ -108,13 +108,13 @@ define perl_backtrace_an_interp
 printf "cxstack=%#x\n", $cxstack
 if $cxstack
 x/128xw $cxstack
-   

Bug#775744: libapp-stacktrace-perl: FTBFS: Failed 1/4 test programs. 1/6 subtests failed.

[Niko Tyni]

On Sun, Dec 04, 2016 at 03:10:03AM +0100, Axel Beckert wrote:
> Chris Lamb wrote:
> > libapp-stacktrace-perl fails to build from source in unstable/amd64 too:

> Just having read
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=814929#10, it sounds
> as if Niko's observation about longer addresses could also have
> triggered the FTBFS you ran into, too, although it's likely
> different code involved. (If so, I'd clone this bug to track the
> recent FTBFS on probably all architectures separately and downgrade
> the old one to important again.)
> 
> Niko: What do you think?

I expect it's the same bug as the mips64el one that was originally
reported here, but there may be other issues on 32-bit architectures.

I don't think it's worth fiddling with clones and severities, just fixing
the immediate one and seeing how it fares after that seems fine. But
YMMV of course.

The problem itself seems to be that the gdb macros in
 lib/App/Stacktrace/perl_backtrace_raw.txt
cast pointers to the C 'int' type, which is broken on 64-bit
architectures. For instance, in 'perl_backtrace_5_12_x':
set $interpreter = (int) Perl_get_context()

I suppose it used to work on amd64 because perl used low memory addresses
so things didn't overflow, but that has changed with the latest perl
upload, presumably due to gcc PIE changes.

Hope this helps,
-- 
Niko Tyni   nt...@debian.org

Bug#775744: libapp-stacktrace-perl: FTBFS: Failed 1/4 test programs. 1/6 subtests failed.

[Axel Beckert]

Hi Chris,

Chris Lamb wrote:
> libapp-stacktrace-perl fails to build from source in unstable/amd64 too:

Yes, but that might be a different bug, as I suspect your experinced
issue (which I can confirm, too, btw.) happens on _all_ architectures.

>From the initial bug report:
> > # Cannot access memory at address 0x200147b0

>From your log:
>   # Cannot access memory at address 0x8ad29718

Just having read
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=814929#10, it sounds
as if Niko's observation about longer addresses could also have
triggered the FTBFS you ran into, too, although it's likely
different code involved. (If so, I'd clone this bug to track the
recent FTBFS on probably all architectures separately and downgrade
the old one to important again.)

Niko: What do you think?

Regards, Axel
-- 
 ,''`.  |  Axel Beckert , http://people.debian.org/~abe/
: :' :  |  Debian Developer, ftp.ch.debian.org Admin
`. `'   |  4096R: 2517 B724 C5F6 CA99 5329  6E61 2FF9 CD59 6126 16B5
  `-|  1024D: F067 EA27 26B9 C3FC 1486  202E C09E 1D89 9593 0EDE

Bug#775744: libapp-stacktrace-perl: FTBFS: Failed 1/4 test programs. 1/6 subtests failed.

[Chris Lamb]

severity 775744 serious
thanks

libapp-stacktrace-perl fails to build from source in unstable/amd64 too:

  […]

  #   'procselfexe' => '"/proc/self/exe"',
  #   'prototype' => 'define',
  #   'ptrsize' => '8',
  #   'quadkind' => '2',
  #   'quadtype' => 'long',
  #   'randbits' => '48',
  #   'randfunc' => 'Perl_drand48',
  #   'random_r_proto' => 'REENTRANT_PROTO_I_St',
  #   'randseedtype' => 'U32',
  #   'ranlib' => ':',
  #   'rd_nodata' => '-1',
  #   'readdir64_r_proto' => 'REENTRANT_PROTO_I_TSR',
  #   'readdir_r_proto' => 'REENTRANT_PROTO_I_TSR',
  #   'revision' => '5',
  #   'rm' => 'rm',
  #   'rm_try' => '/bin/rm -f try try a.out .out try.[cho] try..o core 
core.try* try.core*',
  #   'rmail' => '',
  #   'run' => '',
  #   'runnm' => 'false',
  #   'sGMTIME_max' => '67768036191676799',
  #   'sGMTIME_min' => '-62167219200',
  #   'sLOCALTIME_max' => '67768036191676799',
  #   'sLOCALTIME_min' => '-62167219200',
  #   'sPRIEUldbl' => '"LE"',
  #   'sPRIFUldbl' => '"LF"',
  #   'sPRIGUldbl' => '"LG"',
  #   'sPRIXU64' => '"lX"',
  #   'sPRId64' => '"ld"',
  #   'sPRIeldbl' => '"Le"',
  #   'sPRIfldbl' => '"Lf"',
  #   'sPRIgldbl' => '"Lg"',
  #   'sPRIi64' => '"li"',
  #   'sPRIo64' => '"lo"',
  #   'sPRIu64' => '"lu"',
  #   'sPRIx64' => '"lx"',
  #   'sSCNfldbl' => '"Lf"',
  #   'sched_yield' => 'sched_yield()',
  #   'scriptdir' => '/usr/bin',
  #   'scriptdirexp' => '/usr/bin',
  #   'sed' => 'sed',
  #   'seedfunc' => 'Perl_drand48_init',
  #   'selectminbits' => '64',
  #   'selecttype' => 'fd_set *',
  #   'sendmail' => '',
  #   'setgrent_r_proto' => '0',
  #   'sethostent_r_proto' => '0',
  #   'setlocale_r_proto' => '0',
  #   'setnetent_r_proto' => '0',
  #   'setprotoent_r_proto' => '0',
  #   'setpwent_r_proto' => '0',
  #   'setservent_r_proto' => '0',
  #   'sh' => '/bin/sh',
  #   'shar' => '',
  #   'sharpbang' => '#!',
  #   'shmattype' => 'void *',
  #   'shortsize' => '2',
  #   'shrpenv' => '',
  #   'shsharp' => 'true',
  #   'sig_count' => '65',
  #   'sig_name' => 'ZERO HUP INT QUIT ILL TRAP ABRT BUS FPE KILL USR1 SEGV 
USR2 PIPE ALRM TERM STKFLT CHLD CONT STOP TSTP TTIN TTOU URG XCPU XFSZ VTALRM 
PROF WINCH IO PWR SYS NUM32 NUM33 RTMIN NUM35 NUM36 NUM37 NUM38 NUM39 NUM40 
NUM41 NUM42 NUM43 NUM44 NUM45 NUM46 NUM47 NUM48 NUM49 NUM50 NUM51 NUM52 NUM53 
NUM54 NUM55 NUM56 NUM57 NUM58 NUM59 NUM60 NUM61 NUM62 NUM63 RTMAX IOT CLD POLL 
UNUSED ',
  #   'sig_name_init' => '"ZERO", "HUP", "INT", "QUIT", "ILL", "TRAP", "ABRT", 
"BUS", "FPE", "KILL", "USR1", "SEGV", "USR2", "PIPE", "ALRM", "TERM", "STKFLT", 
"CHLD", "CONT", "STOP", "TSTP", "TTIN", "TTOU", "URG", "XCPU", "XFSZ", 
"VTALRM", "PROF", "WINCH", "IO", "PWR", "SYS", "NUM32", "NUM33", "RTMIN", 
"NUM35", "NUM36", "NUM37", "NUM38", "NUM39", "NUM40", "NUM41", "NUM42", 
"NUM43", "NUM44", "NUM45", "NUM46", "NUM47", "NUM48", "NUM49", "NUM50", 
"NUM51", "NUM52", "NUM53", "NUM54", "NUM55", "NUM56", "NUM57", "NUM58", 
"NUM59", "NUM60", "NUM61", "NUM62", "NUM63", "RTMAX", "IOT", "CLD", "POLL", 
"UNUSED", 0',
  #   'sig_num' => '0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 
23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 
49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 6 17 29 31 ',
  #   'sig_num_init' => '0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 
16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 
36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 
56, 57, 58, 59, 60, 61, 62, 63, 64, 6, 17, 29, 31, 0',
  #   'sig_size' => '69',
  #   'signal_t' => 'void',
  #   'sitearch' => '/usr/local/lib/x86_64-linux-gnu/perl/5.24.1',
  #   'sitearchexp' => '/usr/local/lib/x86_64-linux-gnu/perl/5.24.1',
  #   'sitebin' => '/usr/local/bin',
  #   'sitebinexp' => '/usr/local/bin',
  #   'sitehtml1dir' => '',
  #   'sitehtml1direxp' => '',
  #   'sitehtml3dir' => '',
  #   'sitehtml3direxp' => '',
  #   'sitelib' => '/usr/local/share/perl/5.24.1',
  #   'sitelib_stem' => '',
  #   'sitelibexp' => '/usr/local/share/perl/5.24.1',
  #   'siteman1dir' => '/usr/local/man/man1',
  #   'siteman1direxp' => '/usr/local/man/man1',
  #   'siteman3dir' => '/usr/local/man/man3',
  #   'siteman3direxp' => '/usr/local/man/man3',
  #   'siteprefix' => '/usr/local',
  #   'siteprefixexp' => '/usr/local',
  #   'sitescript' => '/usr/local/bin',
  #   'sitescriptexp' => '/usr/local/bin',
  #   'sizesize' => '8',
  #   'sizetype' => 'size_t',
  #   'sleep' => '',
  #   'smail' => '',
  #   'so' => 'so',
  #   'sockethdr' => '',
  #   'socketlib' => '',
  #   'socksizetype' => 'socklen_t',
  #   'sort' => 'sort',
  #   'spackage' => 'Perl5',
  #   'spitshell' => 'cat',
  #   'srand48_r_proto' => 'REENTRANT_PROTO_I_LS',
  #   'srandom_r_proto' => 'REENTRANT_PROTO_I_TS',
  #   'src' => '.',
  #   'ssizetype' => 'ssize_t',
  #   'st_ino_sign' => '1',
  #   'st_ino_size' => '8',
  #   'startperl' => '#!/usr/bin/perl',
  #   'startsh' => '#!/bin/sh',
  #