Re: Perl problem - loadable library and perl binaries are mismatched
On Tue, Mar 05, 2024 at 11:55:28PM +0100, John Paul Adrian Glaubitz wrote:
> > Thanks! You saved me a lot of headaches!
FWIW to do a bootstrap build of perl I had to:
- disable libdb and libgdbm build-deps
- disable build-time tests, which would pick up old ndbm module from
on-disk and break as a result
Not sure if your bootstrap path was similar.
> I have run into this issue again trying to rebuild libxs-parse-keyword-perl
> with a src:perl that was built with dpkg_1.22.5:
> powerpc-linux-gnu-gcc -Isrc/ -I/usr/lib/powerpc-linux-gnu/perl/5.38/CORE
> -DVERSION="0.39" -DXS_VERSION="0.39" -fPIC -I. -Ihax -c -D_REENTRANT
> -D_GNU_SOURCE -DDEBIAN -fwrapv -fno-strict-aliasing -pipe
> -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -g -O2
> -Werror=implicit-function-declaration
> -ffile-prefix-map=/home/glaubitz/perl-modules/libxs-parse-keyword-perl-0.39=.
> -fstack-
> protector-strong -Wformat -Werror=format-security -g -O2
> -Werror=implicit-function-declaration
> -ffile-prefix-map=/home/glaubitz/perl-modules/libxs-parse-keyword-perl-0.39=.
> -fstack-protector-strong -
> Wformat -Werror=format-security -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
> -D_TIME_BITS=64 -Wdate-time -D_FORTIFY_SOURCE=2 -o lib/XS/Parse/Keyword.o
> lib/XS/Parse/Keyword.c
> ExtUtils::Mkbootstrap::Mkbootstrap('blib/arch/auto/XS/Parse/Keyword/Keyword.bs')
> powerpc-linux-gnu-gcc -g -O2 -Werror=implicit-function-declaration
> -ffile-prefix-map=/home/glaubitz/perl-modules/libxs-parse-keyword-perl-0.39=.
> -fstack-protector-strong -Wformat -Werror=format-
> security -Wl,-z,relro -Wl,-z,now -shared -L/usr/local/lib
> -fstack-protector-strong -o blib/arch/auto/XS/Parse/Keyword/Keyword.so
> lib/XS/Parse/Keyword.o src/infix.o src/keyword.o
> Parser.c: loadable library and perl binaries are mismatched (got first
> handshake key 0xb600080, needed 0xb700080)
> dh_auto_build: error: /usr/bin/perl Build returned exit code 1
> make: *** [debian/rules:6: binary-arch] Error 1
> dpkg-buildpackage: error: debian/rules binary-arch subprocess returned exit
> status 2
I did not run into this. Do you have a mix of old and new packages from
perl source installed?
> Also, I noticed that libxs-parse-keyword-perl build-depends on
> libextutils-cbuilder-perl which is apparently obsolete and also still
> depends on the old Perl API [1] which makes me wonder how
> libxs-parse-keyword-perl was built for armhf and armel [2].
When modules are incorporated into perl, perl declares a provides: for them.
There's no reason to install the libextutils-cbuilder-perl package anymore.
(which, btw, was an arch: all package, so in any case it wouldn't be
affected by the ABI transition.)
--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer https://www.debian.org/
slanga...@ubuntu.com vor...@debian.org
signature.asc
Description: PGP signature
Re: Re: Perl problem - loadable library and perl binaries are mismatched
Hi Roderich,
On Wed, 2024-03-06 at 19:20 +0100, Roderich Schupp wrote:
> Hi,
>
> > Parser.c: loadable library and perl binaries are mismatched (got first
> > handshake key 0xb600080, needed 0xb700080)
> >
>
>
> The upper 16 bits in these keys (i.e. 0xb60 vs 0xb70) is
> sizeof(PerlInterpreter), the one that some XS module saw
> when it was built vs the size your current perl executable was built with.
> From the location of the error message
> it looks as the build process ("perl Build") has just created the "glue"
> shared library (blib/arch/auto/XS/Parse/
> Keyword/Keyword.so), next it is going to generate documentation (man pages).
> Unless there's an error warning,
> this doesn't produce any output. I ran "perl Build" under strace, this shows
> that doc generation loads Pod::Html
> (probably to generate HTML pages as well, though none were requested) and
> finally this loads HTML::Parser. The
> latter is an XS module
> (/usr/lib/x86_64-linux-gnu/perl5/5.38/auto/HTML/Parser/Parser.so) and seems
> to emit the
> above message.
>
> So the reason is that your HTML/Parser/Parser.so (maybe a version not in the
> canonical path?) is built with a
> different struct PerlInterpreter. The difference in sizeof(PerlInterpreter)
> can probably explained with the
> time64 transition as PerlInterpreter contains a struct stat.
Thanks a lot for the detailed analysis. In fact, libhtml-parser-perl has not
been rebuilt against the time64_t
Perl package yet [1] which would align with your explanation. I'll try to
rebuild the package locally and if
it fixes the problem, I'll binNMU it for powerpc.
Your explanation will enable me to debug future occurrences as I now understand
the underlying problem.
Thanks,
Adrian
> [1] https://buildd.debian.org/status/package.php?p=libhtml-parser-perl
--
.''`. John Paul Adrian Glaubitz
: :' : Debian Developer
`. `' Physicist
`-GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
Re: Perl problem - loadable library and perl binaries are mismatched
On Tue, 05 Mar 2024 23:55:28 +0100, John Paul Adrian Glaubitz wrote: > Also, I noticed that libxs-parse-keyword-perl build-depends on > libextutils-cbuilder-perl > which is apparently obsolete and also still depends on the old Perl API [1] > which makes > me wonder how libxs-parse-keyword-perl was built for armhf and armel [2]. > > Adrian > > > [1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1060712 > > [2] > > https://buildd.debian.org/status/package.php?p=libxs-parse-keyword-perl=sid libextutils-cbuilder-perl is (not only a separate package but also) Provided by perl. Cheers, gregor -- .''`. https://info.comodo.priv.at -- Debian Developer https://www.debian.org : :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D 85FA BB3A 6801 8649 AA06 `. `' Member VIBE!AT & SPI Inc. -- Supporter Free Software Foundation Europe `- signature.asc Description: Digital Signature
Re: Perl problem - loadable library and perl binaries are mismatched
Hi,
On Tue, 2024-03-05 at 23:10 +0100, John Paul Adrian Glaubitz wrote:
> >
> > oks like it's built with dpkg-dev_1.22.4 but the time64 build flags are
> > only activated with 1.22.5.
>
> Ah, that would explain it, thank you so much!
>
> > I think there was talk about making them the default in gcc too, not
> > sure if they got there yet.
> >
> > I suppose Perl could/should store them in its configuration so they'd be
> > passed to all XS module builds regardless of what dpkg-buildflags says.
> > But currently things from dpkg-buildflags get explicitly filtered away [1].
> >
> > IIRC the rationale for this was that packages could opt in/out of security
> > hardening flags independently. That doesn't seem desirable here as they
> > make the XS module ABI incompatible as you've noticed.
> >
> > [1] see https://sources.debian.org/src/perl/5.38.2-3.1/debian/rules/#L188
> > I think -fstack-protector gets passed through there as an exception,
> > so doing the same with the relevant time64 flags should do the trick.
> >
>
> Thanks! You saved me a lot of headaches!
I have run into this issue again trying to rebuild libxs-parse-keyword-perl
with a src:perl that was built with dpkg_1.22.5:
Building XS-Parse-Keyword
powerpc-linux-gnu-gcc -Isrc/ -I/usr/lib/powerpc-linux-gnu/perl/5.38/CORE -fPIC
-I. -Ihax -c -D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv -fno-strict-aliasing
-pipe -I/usr/local/include -
D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -g -O2
-Werror=implicit-function-declaration
-ffile-prefix-map=/home/glaubitz/perl-modules/libxs-parse-keyword-perl-0.39=.
-fstack-protector-strong -Wformat -
Werror=format-security -g -O2 -Werror=implicit-function-declaration
-ffile-prefix-map=/home/glaubitz/perl-modules/libxs-parse-keyword-perl-0.39=.
-fstack-protector-strong -Wformat -Werror=format-
security -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_TIME_BITS=64 -Wdate-time
-D_FORTIFY_SOURCE=2 -o src/infix.o src/infix.c
powerpc-linux-gnu-gcc -Isrc/ -I/usr/lib/powerpc-linux-gnu/perl/5.38/CORE -fPIC
-I. -Ihax -c -D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv -fno-strict-aliasing
-pipe -I/usr/local/include -
D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -g -O2
-Werror=implicit-function-declaration
-ffile-prefix-map=/home/glaubitz/perl-modules/libxs-parse-keyword-perl-0.39=.
-fstack-protector-strong -Wformat -
Werror=format-security -g -O2 -Werror=implicit-function-declaration
-ffile-prefix-map=/home/glaubitz/perl-modules/libxs-parse-keyword-perl-0.39=.
-fstack-protector-strong -Wformat -Werror=format-
security -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_TIME_BITS=64 -Wdate-time
-D_FORTIFY_SOURCE=2 -o src/keyword.o src/keyword.c
src/keyword.c: In function 'probe_piece':
src/keyword.c:348:37: warning: format '%d' expects argument of type 'int', but
argument 2 has type 'U32' {aka 'long unsigned int'} [-Wformat=]
348 | croak("TODO: probe_piece on type=%d\n", type);
|~^
| | |
| int U32 {aka long unsigned int}
|%ld
src/keyword.c: In function 'parse_piece':
src/keyword.c:828:37: warning: format '%d' expects argument of type 'int', but
argument 2 has type 'U32' {aka 'long unsigned int'} [-Wformat=]
828 | croak("TODO: parse_piece on type=%d\n", type);
|~^
| | |
| int U32 {aka long unsigned int}
|%ld
powerpc-linux-gnu-gcc -Isrc/ -I/usr/lib/powerpc-linux-gnu/perl/5.38/CORE
-DVERSION="0.39" -DXS_VERSION="0.39" -fPIC -I. -Ihax -c -D_REENTRANT
-D_GNU_SOURCE -DDEBIAN -fwrapv -fno-strict-aliasing -pipe
-I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -g -O2
-Werror=implicit-function-declaration
-ffile-prefix-map=/home/glaubitz/perl-modules/libxs-parse-keyword-perl-0.39=.
-fstack-
protector-strong -Wformat -Werror=format-security -g -O2
-Werror=implicit-function-declaration
-ffile-prefix-map=/home/glaubitz/perl-modules/libxs-parse-keyword-perl-0.39=.
-fstack-protector-strong -
Wformat -Werror=format-security -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
-D_TIME_BITS=64 -Wdate-time -D_FORTIFY_SOURCE=2 -o lib/XS/Parse/Keyword.o
lib/XS/Parse/Keyword.c
ExtUtils::Mkbootstrap::Mkbootstrap('blib/arch/auto/XS/Parse/Keyword/Keyword.bs')
powerpc-linux-gnu-gcc -g -O2 -Werror=implicit-function-declaration
-ffile-prefix-map=/home/glaubitz/perl-modules/libxs-parse-keyword-perl-0.39=.
-fstack-protector-strong -Wformat -Werror=format-
security -Wl,-z,relro -Wl,-z,now -shared -L/usr/local/lib
-fstack-protector-strong -o blib/arch/auto/XS/Parse/Keyword/Keyword.so
lib/XS/Parse/Keyword.o src/infix.o src/keyword.o
Parser.c: loadable library and perl binaries are mismatched (got first
handshake key 0xb600080, needed 0xb700080)
Re: Perl problem - loadable library and perl binaries are mismatched
(Oops, forgot the Cc you asked for. So resending. Apologies for the duplicate on the list.) On Tue, Mar 05, 2024 at 09:17:17PM +0100, John Paul Adrian Glaubitz wrote: > I am getting strange Perl error after rebuilding Perl for the time64_t > transition on powerpc: > > loadable library and perl binaries are mismatched (got first handshake key > 0xb600080, needed 0xb700080) > > See: > https://buildd.debian.org/status/fetch.php?pkg=libdevice-usb-perl=powerpc=0.38-3=1709663348=0 > > I have already rebuilt Perl once again against the new time64_t libraries, > but that didn't help although the package builds fine locally. > > Does anyone knowledgeable with Perl know what's going on? (You're in somewhat uncharted territory unfortunately, as none of this was tested beforehand.) Looks like it's built with dpkg-dev_1.22.4 but the time64 build flags are only activated with 1.22.5. I think there was talk about making them the default in gcc too, not sure if they got there yet. I suppose Perl could/should store them in its configuration so they'd be passed to all XS module builds regardless of what dpkg-buildflags says. But currently things from dpkg-buildflags get explicitly filtered away [1]. IIRC the rationale for this was that packages could opt in/out of security hardening flags independently. That doesn't seem desirable here as they make the XS module ABI incompatible as you've noticed. [1] see https://sources.debian.org/src/perl/5.38.2-3.1/debian/rules/#L188 I think -fstack-protector gets passed through there as an exception, so doing the same with the relevant time64 flags should do the trick. -- Niko Tyni nt...@debian.org
Re: Perl problem - loadable library and perl binaries are mismatched
On Wed, 2024-03-06 at 00:08 +0200, Niko Tyni wrote: > (Oops, forgot the Cc you asked for. So resending. Apologies for the > duplicate on the list.) No worries. > On Tue, Mar 05, 2024 at 09:17:17PM +0100, John Paul Adrian Glaubitz wrote: > > > I am getting strange Perl error after rebuilding Perl for the time64_t > > transition on powerpc: > > > > loadable library and perl binaries are mismatched (got first handshake key > > 0xb600080, needed 0xb700080) > > > > See: > > https://buildd.debian.org/status/fetch.php?pkg=libdevice-usb-perl=powerpc=0.38-3=1709663348=0 > > > > I have already rebuilt Perl once again against the new time64_t libraries, > > but that didn't help although the package builds fine locally. > > > > Does anyone knowledgeable with Perl know what's going on? > > (You're in somewhat uncharted territory unfortunately, as none of this > was tested beforehand.) Yikes. > Looks like it's built with dpkg-dev_1.22.4 but the time64 build flags are > only activated with 1.22.5. Ah, that would explain it, thank you so much! > I think there was talk about making them the default in gcc too, not > sure if they got there yet. > > I suppose Perl could/should store them in its configuration so they'd be > passed to all XS module builds regardless of what dpkg-buildflags says. > But currently things from dpkg-buildflags get explicitly filtered away [1]. > > IIRC the rationale for this was that packages could opt in/out of security > hardening flags independently. That doesn't seem desirable here as they > make the XS module ABI incompatible as you've noticed. > > [1] see https://sources.debian.org/src/perl/5.38.2-3.1/debian/rules/#L188 > I think -fstack-protector gets passed through there as an exception, > so doing the same with the relevant time64 flags should do the trick. > Thanks! You saved me a lot of headaches! Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer `. `' Physicist `-GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
Re: Perl problem - loadable library and perl binaries are mismatched
On Tue, Mar 05, 2024 at 09:17:17PM +0100, John Paul Adrian Glaubitz wrote: > I am getting strange Perl error after rebuilding Perl for the time64_t > transition on powerpc: > > loadable library and perl binaries are mismatched (got first handshake key > 0xb600080, needed 0xb700080) > > See: > https://buildd.debian.org/status/fetch.php?pkg=libdevice-usb-perl=powerpc=0.38-3=1709663348=0 > > I have already rebuilt Perl once again against the new time64_t libraries, > but that didn't help although the package builds fine locally. > > Does anyone knowledgeable with Perl know what's going on? (You're in somewhat uncharted territory unfortunately, as none of this was tested beforehand.) Looks like it's built with dpkg-dev_1.22.4 but the time64 build flags are only activated with 1.22.5. I think there was talk about making them the default in gcc too, not sure if they got there yet. I suppose Perl could/should store them in its configuration so they'd be passed to all XS module builds regardless of what dpkg-buildflags says. But currently things from dpkg-buildflags get explicitly filtered away [1]. IIRC the rationale for this was that packages could opt in/out of security hardening flags independently. That doesn't seem desirable here as they make the XS module ABI incompatible as you've noticed. [1] see https://sources.debian.org/src/perl/5.38.2-3.1/debian/rules/#L188 I think -fstack-protector gets passed through there as an exception, so doing the same with the relevant time64 flags should do the trick. -- Niko Tyni nt...@debian.org
