Re: [RFC] building postgres with meson - perl embedding
Hi,
I was trying to fix a few perl embedding oddities in the meson
patchset.
Whenever I have looked at the existing code, I've been a bit confused about
the following
code/comment in perl.m4:
# PGAC_CHECK_PERL_EMBED_LDFLAGS
# -
# We are after Embed's ldopts, but without the subset mentioned in
# Config's ccdlflags; [...]
pgac_tmp1=`$PERL -MExtUtils::Embed -e ldopts`
pgac_tmp2=`$PERL -MConfig -e 'print $Config{ccdlflags}'`
perl_embed_ldflags=`echo X"$pgac_tmp1" | sed -e "s/^X//" -e
"s%$pgac_tmp2%%" -e ["s/ -arch [-a-zA-Z0-9_]*//g"]`
What is the reason behind subtracting ccdlflags?
The comment originates in:
commit d69a419e682c2d39c2355105a7e5e2b90357c8f0
Author: Tom Lane
Date: 2009-09-08 18:15:55 +
Remove any -arch switches given in ExtUtils::Embed's ldopts from our
perl_embed_ldflags setting. On OS X it seems that ExtUtils::Embed is
trying to force a universal binary to be built, but you need to specify
that a lot further upstream if you want Postgres built that way; the only
result of including -arch in perl_embed_ldflags is some warnings at the
plperl.so link step. Per my complaint and Jan Otto's suggestion.
but the subtraction goes all the way back to
commit 7662419f1bc1a994193c319c9304dfc47e121c98
Author: Peter Eisentraut
Date: 2002-05-28 16:57:53 +
Change PL/Perl and Pg interface build to use configured compiler and
Makefile.shlib system, not MakeMaker.
Greetings,
Andres Freund
Re: [RFC] building postgres with meson - perl embedding
Andres Freund writes:
> What is the reason behind subtracting ccdlflags?
It looks like the coding actually originated here:
commit f5d0c6cad5bb2706e0e63f3f8f32e431ea428100
Author: Bruce Momjian
Date: Wed Jun 20 00:26:06 2001 +
Apparently, on some systems, ExtUtils::Embed and MakeMaker are slightly
broken, and its impossible to make a shared library when compiling with
both CCDLFLAGS and LDDLFAGS, you have to pick one or the other.
Alex Pilosov
and Peter just copied the logic in 7662419f1. Considering that
the point of 7662419f1 was to get rid of MakeMaker, maybe we no
longer needed that at that point.
On my RHEL box, the output of ldopts is sufficiently redundant
that the subtraction doesn't actually accomplish much:
$ perl -MExtUtils::Embed -e ldopts
-Wl,--enable-new-dtags -Wl,-z,relro -Wl,-z,now
-specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,-z,relro -Wl,-z,now
-specs=/usr/lib/rpm/redhat/redhat-hardened-ld -fstack-protector-strong
-L/usr/local/lib -L/usr/lib64/perl5/CORE -lperl -lpthread -lresolv -ldl -lm
-lcrypt -lutil -lc
$ perl -MConfig -e 'print $Config{ccdlflags}'
-Wl,--enable-new-dtags -Wl,-z,relro -Wl,-z,now
-specs=/usr/lib/rpm/redhat/redhat-hardened-ld
which leads to
$ grep perl_embed_ldflags src/Makefile.global
perl_embed_ldflags = -Wl,-z,relro -Wl,-z,now
-specs=/usr/lib/rpm/redhat/redhat-hardened-ld -fstack-protector-strong
-L/usr/local/lib -L/usr/lib64/perl5/CORE -lperl -lpthread -lresolv -ldl -lm
-lcrypt -lutil -lc
so the only thing we actually got rid of was -Wl,--enable-new-dtags,
which I think we'll put back anyway.
Things might be different elsewhere of course, but I'm tempted
to take out the ccdlflags subtraction and see what the buildfarm
says.
regards, tom lane
Re: [RFC] building postgres with meson - perl embedding
I wrote:
> Andres Freund writes:
>> What is the reason behind subtracting ccdlflags?
> It looks like the coding actually originated here:
> commit f5d0c6cad5bb2706e0e63f3f8f32e431ea428100
Ah, here's the thread leading up to that:
https://www.postgresql.org/message-id/flat/200106191206.f5JC6R108371%40candle.pha.pa.us
The use of ldopts rather than hand-hacked link options seems to date to
0ed7864d6, only a couple days before that. I don't think we had a
buildfarm then, but I'd bet against the problem being especially
widespread even then, or more people would've complained.
BTW, the business with zapping arch options seems to not be necessary
anymore either on recent macOS:
$ perl -MExtUtils::Embed -e ldopts
-fstack-protector-strong
-L/System/Library/Perl/5.30/darwin-thread-multi-2level/CORE -lperl
$ perl -MConfig -e 'print $Config{ccdlflags}'
$
(same results on either Intel or ARM Mac). However, it looks like it
is still necessary to keep locust happy, and I have no idea just when
Apple stopped using arch switches here, so we'd better keep that.
regards, tom lane
Re: [RFC] building postgres with meson - perl embedding
Hi, On 2022-02-07 20:42:09 -0500, Tom Lane wrote: > Andres Freund writes: > > What is the reason behind subtracting ccdlflags? > > It looks like the coding actually originated here: > > commit f5d0c6cad5bb2706e0e63f3f8f32e431ea428100 > Author: Bruce Momjian > Date: Wed Jun 20 00:26:06 2001 + > > Apparently, on some systems, ExtUtils::Embed and MakeMaker are slightly > broken, and its impossible to make a shared library when compiling with > both CCDLFLAGS and LDDLFAGS, you have to pick one or the other. > > Alex Pilosov > > and Peter just copied the logic in 7662419f1. Considering that > the point of 7662419f1 was to get rid of MakeMaker, maybe we no > longer needed that at that point. Yea. And maybe what was broken in 2001 isn't broken anymore either ;) Looking at a number of OSs: debian sid: embed: -Wl,-E -fstack-protector-strong -L/usr/local/lib -L/usr/lib/x86_64-linux-gnu/perl/5.34/CORE -lperl -ldl -lm -lpthread -lc -lcrypt ldopts: -Wl,-E fedora: embed: -Wl,--enable-new-dtags -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -fstack-protector-strong -L/usr/local/lib -L/usr/lib64/perl5/CORE -lperl -lpthread -lresolv -ldl -lm -lcrypt -lutil -lc ldopts: -Wl,--enable-new-dtags -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 suse tumbleweed: embed: -Wl,-E -Wl,-rpath,/usr/lib/perl5/5.34.0/x86_64-linux-thread-multi/CORE -L/usr/local/lib64 -fstack-protector-strong -L/usr/lib/perl5/5.34.0/x86_64-linux-thread-multi/CORE -lperl -lm -ldl -lcrypt -lpthread ldopts: -Wl,-E -Wl,-rpath,/usr/lib/perl5/5.34.0/x86_64-linux-thread-multi/CORE freebsd: embed: -Wl,-R/usr/local/lib/perl5/5.30/mach/CORE -pthread -Wl,-E -fstack-protector-strong -L/usr/local/lib -L/usr/local/lib/perl5/5.30/mach/CORE -lperl -lpthread -lm -lcrypt -lutil ldopts: -Wl,-R/usr/local/lib/perl5/5.30/mach/CORE netbsd: embed: -Wl,-E -Wl,-R/usr/pkg/lib/perl5/5.34.0/x86_64-netbsd-thread-multi/CORE -pthread -L/usr/lib -Wl,-R/usr/lib -Wl,-R/usr/pkg/lib -L/usr/pkg/lib -L/usr/pkg/lib/perl5/5.34.0/x86_64-netbsd-thread-multi/CORE -lperl -lm -lcrypt -lpthread ldopts: -Wl,-E -Wl,-R/usr/pkg/lib/perl5/5.34.0/x86_64-netbsd-thread-multi/CORE openbsd: embed: -Wl,-R/usr/libdata/perl5/amd64-openbsd/CORE -Wl,-E -fstack-protector-strong -L/usr/local/lib -L/usr/libdata/perl5/amd64-openbsd/CORE -lperl -lm -lc ldopts: -Wl,-R/usr/libdata/perl5/amd64-openbsd/CORE aix: embed: -bE:/usr/opt/perl5/lib64/5.28.1/aix-thread-multi-64all/CORE/perl.exp -bE:/usr/opt/perl5/lib64/5.28.1/aix-thread-multi-64all/CORE/perl.exp -brtl -bdynamic -b64 -L/usr/opt/perl5/lib64/5.28.1/aix-thread-multi-64all/CORE -lperl -lpthread -lbind -lnsl -ldl -lld -lm -lcrypt -lpthreads -lc ldopts: -bE:/usr/opt/perl5/lib64/5.28.1/aix-thread-multi-64all/CORE/perl.exp -bE:/usr/opt/perl5/lib64/5.28.1/aix-thread-multi-64all/CORE/perl.exp mac m1 monterey: embed: -fstack-protector-strong -L/System/Library/Perl/5.30/darwin-thread-multi-2level/CORE -lperl ldopts: windows msys install ucrt perl: embed: -s -L"C:\dev\msys64\ucrt64\lib\perl5\core_perl\CORE" -L"C:\dev\msys64\ucrt64\lib" "C:\dev\msys64\ucrt64\lib\perl5\core_perl\CORE\libperl532.a" ldopts: windows strawberrry perl: embed: -s -L"C:\STRAWB~1\perl\lib\CORE" -L"C:\STRAWB~1\c\lib" "C:\STRAWB~1\perl\lib\CORE\libperl530.a" "C:\STRAWB~1\c\x86_64-w64-mingw32\lib\libmoldname.a" "C:\STRAWB~1\c\x86_64-w64-mingw32\lib\libkernel32.a" "C:\STRAWB~1\c\x86_64-w64-mingw32\lib\libuser32.a" "C:\STRAWB~1\c\x86_64-w64-mingw32\lib\libgdi32.a" "C:\STRAWB~1\c\x86_64-w64-mingw32\lib\libwinspool.a" "C:\STRAWB~1\c\x86_64-w64-mingw32\lib\libcomdlg32.a" "C:\STRAWB~1\c\x86_64-w64-mingw32\lib\libadvapi32.a" "C:\STRAWB~1\c\x86_64-w64-mingw32\lib\libshell32.a" "C:\STRAWB~1\c\x86_64-w64-mingw32\lib\libole32.a" "C:\STRAWB~1\c\x86_64-w64-mingw32\lib\liboleaut32.a" "C:\STRAWB~1\c\x86_64-w64-mingw32\lib\libnetapi32.a" "C:\STRAWB~1\c\x86_64-w64-mingw32\lib\libuuid.a" "C:\STRAWB~1\c\x86_64-w64-mingw32\lib\libws2_32.a" "C:\STRAWB~1\c\x86_64-w64-mingw32\lib\libmpr.a" "C:\STRAWB~1\c\x86_64-w64-mingw32\lib\libwinmm.a" "C:\STRAWB~1\c\x86_64-w64-mingw32\lib\libversion.a" "C:\STRAWB~1\c\x86_64-w64-mingw32\lib\libodbc32.a" "C:\STRAWB~1\c\x86_64-w64-mingw32\lib\libodbccp32.a" "C:\STRAWB~1\c\x86_64-w64-mingw32\lib\libcomctl32.a" ldopts: So on windows, macos it makes no difference because ldopts is empty. On various linuxes, except red-hat and debian ones, as well as on the BSDs, it removes rpath. Which we then add back in various places (pl and transform modules). On debian the added rpath never will contain the library. AIX is the one exception. Specifying -bE... doesn't seem right for building plperl etc. S
Re: [RFC] building postgres with meson - perl embedding
On 2/7/22 21:40, Tom Lane wrote: > I wrote: >> Andres Freund writes: >>> What is the reason behind subtracting ccdlflags? >> It looks like the coding actually originated here: >> commit f5d0c6cad5bb2706e0e63f3f8f32e431ea428100 > Ah, here's the thread leading up to that: > > https://www.postgresql.org/message-id/flat/200106191206.f5JC6R108371%40candle.pha.pa.us > > The use of ldopts rather than hand-hacked link options seems to date to > 0ed7864d6, only a couple days before that. I don't think we had a > buildfarm then, but I'd bet against the problem being especially > widespread even then, or more people would've complained. The buildfarm's first entry is from 22 Oct 2004. cheers andrew -- Andrew Dunstan EDB: https://www.enterprisedb.com
Re: [RFC] building postgres with meson - perl embedding
Andres Freund writes: > On 2022-02-07 20:42:09 -0500, Tom Lane wrote: >> ... Peter just copied the logic in 7662419f1. Considering that >> the point of 7662419f1 was to get rid of MakeMaker, maybe we no >> longer needed that at that point. > Yea. And maybe what was broken in 2001 isn't broken anymore either ;) Yeah --- note that Bruce was complaining about a problem on Perl 5.005, which was already a bit over-the-hill in 2001. > AIX is the one exception. Specifying -bE... doesn't seem right for building > plperl etc. So possibly the subtraction accidentally does work for us there... I tried this on AIX 7.2 (using the gcc farm, same build options as hoverfly). The build still works and passes regression tests, but you get a warning about each symbol exported by Perl itself: ... ld: 0711-415 WARNING: Symbol PL_veto_cleanup is already exported. ld: 0711-415 WARNING: Symbol PL_warn_nl is already exported. ld: 0711-415 WARNING: Symbol PL_warn_nosemi is already exported. ld: 0711-415 WARNING: Symbol PL_warn_reserved is already exported. ld: 0711-415 WARNING: Symbol PL_warn_uninit is already exported. ld: 0711-415 WARNING: Symbol PL_WB_invlist is already exported. ld: 0711-415 WARNING: Symbol PL_XPosix_ptrs is already exported. ld: 0711-415 WARNING: Symbol PL_Yes is already exported. ld: 0711-415 WARNING: Symbol PL_Zero is already exported. So there's about 1200 such warnings for plperl, and then the same again for each contrib foo_plperl module. Maybe that's annoying enough that we should keep the logic. OTOH, it seems entirely accidental that it has that effect. I'd be a little inclined to replace it with some rule about stripping '-bE:' switches out of the ldopts result. regards, tom lane
Re: [RFC] building postgres with meson - perl embedding
Hi, On 2022-02-08 18:42:33 -0500, Tom Lane wrote: > I'd be a little inclined to replace it with some rule about stripping '-bE:' > switches out of the ldopts result. Similar. That's a lot easier to understand than than -bE ending up stripped by what we're doing. Should I do so, or do you want to? Greetings, Andres Freund
Re: [RFC] building postgres with meson - perl embedding
Andres Freund writes: > On 2022-02-08 18:42:33 -0500, Tom Lane wrote: >> I'd be a little inclined to replace it with some rule about stripping '-bE:' >> switches out of the ldopts result. > Similar. That's a lot easier to understand than than -bE ending up stripped by > what we're doing. Should I do so, or do you want to? I could look at it later, but if you want to do it, feel free. regards, tom lane
