Bug#909396: systemd: FTBFS on hppa and x32 - relocation can not be used when making a shared object
On 2018-11-18 3:08 p.m., Michael Biebl wrote: It's not at all clear how we get both the -fPIE and -fPIC options in various compilations. The don't seem to be explicitly specified in the systemd package. Do you know? Unfortunately not. You could talk to Jussi Pakkanen, our meson maintainer and upstream of meson, maybe he can help here. Most of the problem is in systemd's top meson.build file: # the oss-fuzz fuzzers are not built with -fPIE, so don't # enable it when we are linking against them if not fuzzer_build possible_cc_flags += '-fPIE' possible_link_flags += '-pie' endif This is a highly questionable method of enabling PIE support in the presence of files where -fPIC may be used. It also adds -pie to link commands with -shared. Since 0.49.0, meson has a b_pie option. Another option that is relevant here is b_staticpic. I don't know if they can work together. It looks like the merging fix in gcc will be backported to gcc-8. As a work around, you could disable LTO on hppa by setting 'b_lto=false' in debian/rules. Dave -- John David Anglin dave.ang...@bell.net
Bug#909396: systemd: FTBFS on hppa and x32 - relocation can not be used when making a shared object
On 2018-11-18 3:08 p.m., Michael Biebl wrote: Unfortunately not. You could talk to Jussi Pakkanen, our meson maintainer and upstream of meson, maybe he can help here. I created two bug reports: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=914543 https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88185 Dave -- John David Anglin dave.ang...@bell.net
Bug#909396: systemd: FTBFS on hppa and x32 - relocation can not be used when making a shared object
Am 18.11.18 um 20:46 schrieb John David Anglin: > On 2018-11-18 1:54 p.m., Michael Biebl wrote: >> Thanks for your detailed analysis. >> If there is something to fix on the systemd side, it would be great if >> you can create a pull request upstream as suggested in >> https://github.com/systemd/systemd/issues/10548 > There's definitely something that needs fixing but I don't know what it > is. It could be in the > debian build infrastructure or ninja. It's not at all clear how we get > both the -fPIE and -fPIC > options in various compilations. The don't seem to be explicitly > specified in the systemd > package. Do you know? Unfortunately not. You could talk to Jussi Pakkanen, our meson maintainer and upstream of meson, maybe he can help here. Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? signature.asc Description: OpenPGP digital signature
Bug#909396: systemd: FTBFS on hppa and x32 - relocation can not be used when making a shared object
On 2018-11-18 1:54 p.m., Michael Biebl wrote: Thanks for your detailed analysis. If there is something to fix on the systemd side, it would be great if you can create a pull request upstream as suggested in https://github.com/systemd/systemd/issues/10548 There's definitely something that needs fixing but I don't know what it is. It could be in the debian build infrastructure or ninja. It's not at all clear how we get both the -fPIE and -fPIC options in various compilations. The don't seem to be explicitly specified in the systemd package. Do you know? Dave -- John David Anglin dave.ang...@bell.net
Bug#909396: systemd: FTBFS on hppa and x32 - relocation can not be used when making a shared object
Hi John, Am 18.11.18 um 19:18 schrieb John David Anglin: > I understand the problem. The merge comment shown above is from gcc-9. > Systemd compiles fine > on hppa using gcc-9. > > The merging of PIC and PIE options is not supported using gcc-8. This is > the corresponding comment in gcc-8: [..] > -fPIE and -fPIC are very similar but the former is intended to be used > for code intended for position > independent executables and the latter for shared libraries. Only one > should be specified in any compilation. > For PIE executables, only -fPIE should be used. If -fPIE, -fpie, -fPIC, > -fpic are specified multiple times, > the last one is selected. Thanks for your detailed analysis. If there is something to fix on the systemd side, it would be great if you can create a pull request upstream as suggested in https://github.com/systemd/systemd/issues/10548 Once that is merged, I'd be happy to cherry-pick any necessary changes. Regards, Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? signature.asc Description: OpenPGP digital signature
Bug#909396: systemd: FTBFS on hppa and x32 - relocation can not be used when making a shared object
On 2018-11-14 2:19 p.m., John David Anglin wrote: The LTO plugin merges the PIC and PIE options used for the compiled objects: /* Merge PIC options: -fPIC + -fpic = -fpic -fPIC + -fno-pic = -fno-pic -fpic/-fPIC + nothin = nothing. It is a common mistake to mix few -fPIC compiled objects into otherwise non-PIC code. We do not want to build everything with PIC then. Similarly we merge PIE options, however in addition we keep -fPIC + -fPIE = -fPIE -fpic + -fPIE = -fpie -fPIC/-fpic + -fpie = -fpie It would be good to warn on mismatches, but it is bit hard to do as we do not know what nothing translates to. */ As noted above, the most common mistake is to miss a -fPIC or -fPIE option in one of the objects. Adding -fPIE to the PIE link options causes the LTO plugin to use it for the compilations it does. Given that it works, implies that all objects in the failing link are in fact PIE, but for some reason the option determination has failed. I understand the problem. The merge comment shown above is from gcc-9. Systemd compiles fine on hppa using gcc-9. The merging of PIC and PIE options is not supported using gcc-8. This is the corresponding comment in gcc-8: /* Merge PIC options: -fPIC + -fpic = -fpic -fPIC + -fno-pic = -fno-pic -fpic/-fPIC + nothin = nothing. It is a common mistake to mix few -fPIC compiled objects into otherwise non-PIC code. We do not want to build everything with PIC then. It would be good to warn on mismatches, but it is bit hard to do as we do not know what nothing translates to. */ If one looks at one of the failing links, [422/1547] cc -o src/udev/ata_id 'src/udev/src@udev@@ata_id@exe/ata_id_ata_id.c.o' -flto -Wl,--no-undefined -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -pie -Wl,--gc-sections -g -O2 -fdebug-prefix-map=/<>=. -Wformat -Werror=format-security -Wl,--start-group src/udev/libudev_static.a src/shared/libsystemd-shared-239.a src/libsystemd/libsystemd_static.a src/basic/libbasic.a src/udev/libudev-basic.a -lrt /usr/lib/hppa-linux-gnu/libcap.so -lacl /usr/lib/hppa-linux-gnu/libcryptsetup.so -lgcrypt /usr/lib/hppa-linux-gnu/libip4tc.so /usr/lib/hppa-linux-gnu/libip6tc.so /usr/lib/hppa-linux-gnu/libselinux.so /usr/lib/hppa-linux-gnu/libidn.so /usr/lib/hppa-linux-gnu/liblzma.so /usr/lib/hppa-linux-gnu/liblz4.so /usr/lib/hppa-linux-gnu/libblkid.so -lrt /usr/lib/hppa-linux-gnu/libmount.so -Wl,--end-group -pthread '-Wl,-rpath,$ORIGIN/:$ORIGIN/../shared:$ORIGIN/../libsystemd:$ORIGIN/../basic' -Wl,-rpath-link,/<>/build-deb/src/udev:/<>/build-deb/src/shared:/<>/build-deb/src/libsystemd:/<>/build-deb/src/basic one sees that ata_id_ata_id.c.o was compiled with just -fPIE: [393/1547] cc -Isrc/udev/src@udev@@scsi_id@exe -Isrc/udev -I../src/udev -Isrc/basic -I../src/basic -Isrc/shared -I../src/shared -Isrc/systemd -I../src/systemd -Isrc/journal -I../src/journal -Isrc/journal-remote -I../src/journal-remote -Isrc/nspawn -I../src/nspawn -Isrc/resolve -I../src/resolve -Isrc/timesync -I../src/timesync -I../src/time-wait-sync -Isrc/login -I../src/login -Isrc/libudev -I../src/libudev -Isrc/core -I../src/core -I../src/libsystemd/sd-bus -I../src/libsystemd/sd-device -I../src/libsystemd/sd-hwdb -I../src/libsystemd/sd-id128 -I../src/libsystemd/sd-netlink -I../src/libsystemd/sd-network -Isrc/libsystemd-network -I../src/libsystemd-network -I. -I../ -flto -fdiagnostics-color=always -pipe -D_FILE_OFFSET_BITS=64 -std=gnu99 -Wextra -Werror=undef -Wlogical-op -Wmissing-include-dirs -Wold-style-definition -Wpointer-arith -Winit-self -Wfloat-equal -Wsuggest-attribute=noreturn -Werror=missing-prototypes -Werror=implicit-function-declaration -Werror=missing-declarations -Werror=return-type -Werror=incompatible-pointer-types -Werror=format=2 -Wstrict-prototypes -Wredundant-decls -Wmissing-noreturn -Wimplicit-fallthrough=5 -Wshadow -Wendif-labels -Wstrict-aliasing=2 -Wwrite-strings -Werror=overflow -Werror=shift-count-overflow -Werror=shift-overflow=2 -Wdate-time -Wnested-externs -ffast-math -fno-common -fdiagnostics-show-option -fno-strict-aliasing -fvisibility=hidden -fstack-protector -fstack-protector-strong --param=ssp-buffer-size=4 -fPIE -ffunction-sections -fdata-sections -Wno-unused-parameter -Wno-missing-field-initializers -Wno-unused-result -Wno-format-signedness -Wno-error=nonnull -Werror=shadow -include config.h -g -O2 -fdebug-prefix-map=/<>=. -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -DLOG_REALM=LOG_REALM_UDEV -MD -MQ 'src/udev/src@udev@@scsi_id@exe/scsi_id_scsi_serial.c.o' -MF 'src/udev/src@udev@@scsi_id@exe/scsi_id_scsi_serial.c.o.d' -o 'src/udev/src@udev@@scsi_id@exe/scsi_id_scsi_serial.c.o' -c ../src/udev/scsi_id/scsi_serial.c However, the objects in the archives used in the link are compiled with both -fPIE and -fPIC. -fPIC is last in the argument list. For example, [259/1547] cc
Bug#909396: systemd: FTBFS on hppa and x32 - relocation can not be used when making a shared object
Hi Michael,
On 2018-11-14 7:38 a.m., Michael Biebl wrote:
If you want to see this fixed, it would be great if you can follow-up at
https://github.com/systemd/systemd/issues/10548#issuecomment-438619429
I don't feel comfortable creating such a pull request myself, as I
wouldn't know what to write in the commit message why the changes are
necessary (and apparently on very specific architectures).
It would thus be best if you can create that PR yourself or at least
send me a patch with a proper commit message and created via "git
format-patch" which I can forward.
I looked at the issue a bit more. One reason this bug may be arch
specific is some
architectures generate position independent code by default.
I agree this fix shouldn't be necessary. The LTO plugin is supposed to
automatically determine
the options needed for the compilations that it does from the options
used for the compiled
options included in the link. See gcc -flinker-output option:
@item -flinker-output=@var{type}
@opindex -flinker-output
This option controls the code generation of the link time optimizer. By
default the linker output is determined by the linker plugin
automatically. For
debugging the compiler and in the case of incremental linking to non-lto
object
file is desired, it may be useful to control the type manually.
The LTO plugin merges the PIC and PIE options used for the compiled objects:
/* Merge PIC options:
-fPIC + -fpic = -fpic
-fPIC + -fno-pic = -fno-pic
-fpic/-fPIC + nothin = nothing.
It is a common mistake to mix few -fPIC compiled objects into
otherwise
non-PIC code. We do not want to build everything with PIC then.
Similarly we merge PIE options, however in addition we keep
-fPIC + -fPIE = -fPIE
-fpic + -fPIE = -fpie
-fPIC/-fpic + -fpie = -fpie
It would be good to warn on mismatches, but it is bit hard to do as
we do not know what nothing translates to. */
As noted above, the most common mistake is to miss a -fPIC or -fPIE
option in one of the objects.
Adding -fPIE to the PIE link options causes the LTO plugin to use it for
the compilations it does. Given
that it works, implies that all objects in the failing link are in fact
PIE, but for some reason the option
determination has failed.
I also see in this build
https://buildd.debian.org/status/fetch.php?pkg=systemd=hppa=239-11=1540749653=0
that the libudev_static.a archive seems to be empty:
[389/1547] rm -f src/udev/libudev_static.a && gcc-ar csrD
src/udev/libudev_static.a
Possibly, this confuses the LTO plugin on hppa. Why do we link against
an empty archive?
On the other hand, libudev-basic.a has objects:
[391/1547] rm -f src/udev/libudev-basic.a && gcc-ar csrD
src/udev/libudev-basic.a
'src/udev/src@udev@@udev-basic@sta/.._libudev_libudev.c.o'
'src/udev/src@udev@@udev-basic@sta/.._libudev_libudev-list.c.o'
'src/udev/src@udev@@udev-basic@sta/.._libudev_libudev-util.c.o'
'src/udev/src@udev@@udev-basic@sta/.._libudev_libudev-device.c.o'
'src/udev/src@udev@@udev-basic@sta/.._libudev_libudev-device-private.c.o'
'src/udev/src@udev@@udev-basic@sta/.._libudev_libudev-enumerate.c.o'
'src/udev/src@udev@@udev-basic@sta/.._libudev_libudev-monitor.c.o'
'src/udev/src@udev@@udev-basic@sta/.._libudev_libudev-queue.c.o'
'src/udev/src@udev@@udev-basic@sta/.._libudev_libudev-hwdb.c.o'
If it can be shown that all objects being linked into the failing links,
[422/1547] cc -o src/udev/ata_id
'src/udev/src@udev@@ata_id@exe/ata_id_ata_id.c.o' -flto
-Wl,--no-undefined -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -pie
-Wl,--gc-sections -g -O2 -fdebug-prefix-map=/<>=. -Wformat
-Werror=format-security -Wl,--start-group src/udev/libudev_static.a
src/shared/libsystemd-shared-239.a src/libsystemd/libsystemd_static.a
src/basic/libbasic.a src/udev/libudev-basic.a -lrt
/usr/lib/hppa-linux-gnu/libcap.so -lacl
/usr/lib/hppa-linux-gnu/libcryptsetup.so -lgcrypt
/usr/lib/hppa-linux-gnu/libip4tc.so /usr/lib/hppa-linux-gnu/libip6tc.so
/usr/lib/hppa-linux-gnu/libselinux.so /usr/lib/hppa-linux-gnu/libidn.so
/usr/lib/hppa-linux-gnu/liblzma.so /usr/lib/hppa-linux-gnu/liblz4.so
/usr/lib/hppa-linux-gnu/libblkid.so -lrt
/usr/lib/hppa-linux-gnu/libmount.so -Wl,--end-group -pthread
'-Wl,-rpath,$ORIGIN/:$ORIGIN/../shared:$ORIGIN/../libsystemd:$ORIGIN/../basic'
-Wl,-rpath-link,/<>/build-deb/src/udev:/<>/build-deb/src/shared:/<>/build-deb/src/libsystemd:/<>/build-deb/src/basic
FAILED: src/udev/ata_id
cc -o src/udev/ata_id 'src/udev/src@udev@@ata_id@exe/ata_id_ata_id.c.o'
-flto -Wl,--no-undefined -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -pie
-Wl,--gc-sections -g -O2 -fdebug-prefix-map=/<>=. -Wformat
-Werror=format-security -Wl,--start-group src/udev/libudev_static.a
src/shared/libsystemd-shared-239.a src/libsystemd/libsystemd_static.a
src/basic/libbasic.a src/udev/libudev-basic.a -lrt
/usr/lib/hppa-linux-gnu/libcap.so -lacl
/usr/lib/hppa-linux-gnu/libcryptsetup.so -lgcrypt
Bug#909396: systemd: FTBFS on hppa and x32 - relocation can not be used when making a shared object
Hi Dave, On Sun, 28 Oct 2018 16:16:52 -0400 John David Anglin wrote: > On 2018-09-25 10:15 AM, John David Anglin wrote: > > I will test the proposed change once this bug is fixed: > > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=909440 > I had a successful build of systemd 239-11 with the attached change. See: > https://buildd.debian.org/status/fetch.php?pkg=systemd=hppa=239-11=1540757105=0 > If you want to see this fixed, it would be great if you can follow-up at https://github.com/systemd/systemd/issues/10548#issuecomment-438619429 I don't feel comfortable creating such a pull request myself, as I wouldn't know what to write in the commit message why the changes are necessary (and apparently on very specific architectures). It would thus be best if you can create that PR yourself or at least send me a patch with a proper commit message and created via "git format-patch" which I can forward. Regards, Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? signature.asc Description: OpenPGP digital signature
Bug#909396: systemd: FTBFS on hppa and x32 - relocation can not be used when making a shared object
On 2018-09-25 10:15 AM, John David Anglin wrote: I will test the proposed change once this bug is fixed: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=909440 I had a successful build of systemd 239-11 with the attached change. See: https://buildd.debian.org/status/fetch.php?pkg=systemd=hppa=239-11=1540757105=0 Dave -- John David Anglin dave.ang...@bell.net Index: systemd-239/meson.build === --- systemd-239.orig/meson.build +++ systemd-239/meson.build @@ -341,7 +341,7 @@ possible_link_flags = [ # enable it when we are linking against them if not fuzzer_build possible_cc_flags += '-fPIE' -possible_link_flags += '-pie' +possible_link_flags += [ '-fPIE', '-pie', ] endif if cc.get_id() == 'clang'
Bug#909396: systemd: FTBFS on hppa and x32 - relocation can not be used when making a shared object
On 2018-09-25 10:15 AM, John David Anglin wrote: I will test the proposed change once this bug is fixed: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=909440 This build was done with the proposed change: https://buildd.debian.org/status/fetch.php?pkg=systemd=hppa=239-10=1537925259=0 Unfortunately, the build was done when the system was under some additional load and there was a timeout in the test-systemd-tmpfiles test: 100/312 test-systemd-tmpfiles TIMEOUT 31.45 s --- command --- /<>/src/test/test-systemd-tmpfiles.py /<>/build-deb/systemd-tmpfiles --- stderr --- [:1] Syntax error. [:1] Failed to parse line: Invalid argument [:1] Path 'closed' not absolute. [:1] Unknown command type 'Y'. [:1] Write file requires argument. [:1] Syntax error. [:1] Unknown modifiers in command 'f!!' [:1] Unknown modifiers in command 'f++' [:1] Unknown modifiers in command 'f+!+' [:1] Unknown modifiers in command 'f!+!' [:1] Failed to substitute specifiers in argument: Invalid slot [:1] Failed to substitute specifiers in argument: Invalid slot [:1] Failed to replace specifiers: /unresolved/filename/%Y [:1] Failed to replace specifiers: /unresolved/filename/sandwich/%v%Y%v [:1] Path '-' not absolute. [:1] Source path is not absolute. [:1] Path 'non/absolute/path' not absolute. [:1] Device file requires argument. [:1] Device file requires argument. [:1] Device file requires argument. [:1] Device file requires argument. [:1] Set extended attribute requires argument. [:1] Set extended attribute requires argument. [:1] Set ACLs requires argument. [:1] Set ACLs requires argument. [:1] Set file attribute requires argument. [:1] Set file attribute requires argument. [:1] Syntax error. [:1] Failed to parse line: Invalid argument [:1] Path 'closed' not absolute. [:1] Unknown command type 'Y'. [:1] Write file requires argument. [:1] Syntax error. [:1] Unknown modifiers in command 'f!!' [:1] Unknown modifiers in command 'f++' [:1] Unknown modifiers in command 'f+!+' [:1] Unknown modifiers in command 'f!+!' [:1] Failed to substitute specifiers in argument: Invalid slot [:1] Failed to substitute specifiers in argument: Invalid slot [:1] Failed to replace specifiers: /unresolved/filename/%Y [:1] Failed to replace specifiers: /unresolved/filename/sandwich/%v%Y%v [:1] Path '-' not absolute. [:1] Source path is not absolute. [:1] Path 'non/absolute/path' not absolute. [:1] Device file requires argument. [:1] Device file requires argument. [:1] Device file requires argument. [:1] Device file requires argument. [:1] Set extended attribute requires argument. [:1] Set extended attribute requires argument. [:1] Set ACLs requires argument. [:1] Set ACLs requires argument. [:1] Set file attribute requires argument. [:1] Set file attribute requires argument. [:1] Failed to resolve specifier: Required $XDG_... variable not defined, skipping All rules containing unresolvable specifiers will be skipped. Did a quick look for the timeout setting but didn't find it last night. This test took 17.77 s on the last successful build: https://buildd.debian.org/status/fetch.php?pkg=systemd=hppa=239-7=1532274327=0 Sibaris is 20% faster than mx3210. So, I tend to think timeout needs adjustment on hppa. Dave -- John David Anglin dave.ang...@bell.net
Bug#909396: systemd: FTBFS on hppa and x32 - relocation can not be used when making a shared object
On 2018-09-25 7:07 AM, Michael Biebl wrote: I'm fine with cherry-picking the patch once it has been reviewed and applied upstream. Would you be willing to create a pull request at https://github.com/systemd/systemd/pulls I will test the proposed change once this bug is fixed: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=909440 It breaks systemd build on hppa: https://buildd.debian.org/status/fetch.php?pkg=systemd=hppa=239-9=1537836912=0 I don't personally have a system that is not behind a firewall. Maybe I could setup something on one of the other hppa buildd machines. There is a hppa porter box which could be used: https://db.debian.org/machines.cgi?host=panama Regards, Dave -- John David Anglin dave.ang...@bell.net
Bug#909396: systemd: FTBFS on hppa and x32 - relocation can not be used when making a shared object
Hi John Am 23.09.18 um 22:27 schrieb John David Anglin: > The attached patch fixes the failing link command although I'm not sure > it's optimal as > generally -fPIE and -pie need to be specified together. Meson appears > to test the options > separately. Putting them together as one option doesn't work. The > systemd build completes > with the patch. I'm fine with cherry-picking the patch once it has been reviewed and applied upstream. Would you be willing to create a pull request at https://github.com/systemd/systemd/pulls Regards, Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? signature.asc Description: OpenPGP digital signature
Bug#909396: systemd: FTBFS on hppa and x32 - relocation can not be used when making a shared object
On 2018-09-23 4:56 PM, Michael Biebl wrote:
Am 23.09.18 um 22:44 schrieb Michael Biebl:
Looking at the "ld" man page, it doesn't say anything about -fPIE, only
about -pie.
Can you please provide some references why on hppa the linker needs
-fPIE in addition to -pie
Fwiw, this command succeeded for 239-7
[425/1545] cc -o src/udev/ata_id
'src/udev/src@udev@@ata_id@exe/ata_id_ata_id.c.o' -flto
-Wl,--no-undefined -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -pie
-Wl,--gc-sections -g -O2 -fdebug-prefix-map=/<>=. -Wformat
-Werror=format-security -Wl,--start-group src/udev/libudev_static.a
src/shared/libsystemd-shared-239.a src/libsystemd/libsystemd_static.a
src/basic/libbasic.a src/udev/libudev-basic.a -lrt
/usr/lib/hppa-linux-gnu/libcap.so -lacl
/usr/lib/hppa-linux-gnu/libcryptsetup.so -lgcrypt
/usr/lib/hppa-linux-gnu/libip4tc.so /usr/lib/hppa-linux-gnu/libip6tc.so
/usr/lib/hppa-linux-gnu/libselinux.so /usr/lib/hppa-linux-gnu/libidn.so
/usr/lib/hppa-linux-gnu/liblzma.so /usr/lib/hppa-linux-gnu/liblz4.so
/usr/lib/hppa-linux-gnu/libblkid.so -lrt
/usr/lib/hppa-linux-gnu/libmount.so -Wl,--end-group
Why is this now suddenly failing on hppa?
This is difficult to say as it's no longer possible to redo the 239-7
without the exact same library
versions. The meson package is now updated in sid.
Here is the gcc documentation for -flto:
@item -flto[=@var{n}]
@opindex flto
This option runs the standard link-time optimizer. When invoked
with source code, it generates GIMPLE (one of GCC's internal
representations) and writes it to special ELF sections in the object
file. When the object files are linked together, all the function
bodies are read from these ELF sections and instantiated as if they
had been part of the same translation unit.
To use the link-time optimizer, @option{-flto} and optimization
options should be specified at compile time and during the final link.
*It is recommended that you compile all the files participating in the**
**same link with the same options and also specify those options at**
**link time.*
For example:
@smallexample
gcc -c -O2 -flto foo.c
gcc -c -O2 -flto bar.c
gcc -o myprog -flto -O2 foo.o bar.o
@end smallexample
The first two invocations to GCC save a bytecode representation
of GIMPLE into special ELF sections inside @file{foo.o} and
@file{bar.o}. The final invocation reads the GIMPLE bytecode from
@file{foo.o} and @file{bar.o}, merges the two files into a single
internal image, and compiles the result as usual. Since both
@file{foo.o} and @file{bar.o} are merged into a single image, this
causes all the interprocedural analyses and optimizations in GCC to
work across the two files as if they were a single one. This means,
for example, that the inliner is able to inline functions in
@file{bar.o} into functions in @file{foo.o} and vice-versa.
Another (simpler) way to enable link-time optimization is:
@smallexample
gcc -o myprog -flto -O2 foo.c bar.c
@end smallexample
The above generates bytecode for @file{foo.c} and @file{bar.c},
merges them together into a single GIMPLE representation and optimizes
them as usual to produce @file{myprog}.
The only important thing to keep in mind is that to enable link-time
optimizations you need to use the GCC driver to perform the link step.
GCC then automatically performs link-time optimization if any of the
objects involved were compiled with the @option{-flto} command-line option.
You generally
should specify the optimization options to be used for link-time
optimization though GCC tries to be clever at guessing an
optimization level to use from the options used at compile time
if you fail to specify one at link time. You can always override
the automatic decision to do link-time optimization
by passing @option{-fno-lto} to the link command.
To make whole program optimization effective, it is necessary to make
certain whole program assumptions. The compiler needs to know
what functions and variables can be accessed by libraries and runtime
outside of the link-time optimized unit. When supported by the linker,
the linker plugin (see @option{-fuse-linker-plugin}) passes information
to the compiler about used and externally visible symbols. When
the linker plugin is not available, @option{-fwhole-program} should be
used to allow the compiler to make these assumptions, which leads
to more aggressive optimization decisions.
When @option{-fuse-linker-plugin} is not enabled, when a file is
compiled with @option{-flto}, the generated object file is larger than
a regular object file because it contains GIMPLE bytecodes and the usual
final code (see @option{-ffat-lto-objects}. This means that
object files with LTO information can be linked as normal object
files; if @option{-fno-lto} is passed to the linker, no
interprocedural optimizations are applied. Note that when
@option{-fno-fat-lto-objects} is enabled the compile stage is faster
but you cannot perform a regular, non-LTO link on them.
...
--
John David Anglin
Bug#909396: systemd: FTBFS on hppa and x32 - relocation can not be used when making a shared object
On 2018-09-23 5:03 PM, Michael Biebl wrote: Am 23.09.18 um 23:00 schrieb John David Anglin: On 2018-09-23 4:44 PM, Michael Biebl wrote: Can you please provide some references why on hppa the linker needs -fPIE in addition to -pie -fPIE is not a "ld" option. It is a compiler option. Right, that's why I don't see why it should be added to possible_link_flags It's needed because the compiler driver is being used to optimize the link. ld is not being used directly. The linker optimization involves various compilations which need the option. The libtool driver is another example of this. If ld were used directly, then the option wouldn't be needed. But then the build process would have to select the correct startup files, etc. -- John David Anglin dave.ang...@bell.net
Bug#909396: systemd: FTBFS on hppa and x32 - relocation can not be used when making a shared object
Am 23.09.18 um 23:00 schrieb John David Anglin: > On 2018-09-23 4:44 PM, Michael Biebl wrote: >> Can you please provide some references why on hppa the linker needs >> -fPIE in addition to -pie > -fPIE is not a "ld" option. It is a compiler option. Right, that's why I don't see why it should be added to possible_link_flags -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? signature.asc Description: OpenPGP digital signature
Bug#909396: systemd: FTBFS on hppa and x32 - relocation can not be used when making a shared object
On 2018-09-23 4:44 PM, Michael Biebl wrote: Am 23.09.18 um 22:27 schrieb John David Anglin: Well the problem was latent in 239-7 and probably gcc changed. The thing is the error points to an error in the link command. On hppa, it is almost always necessary to link shared and pie objects with -fPIC or -fPIE, respectively. There are compilations in the link process and the generated objects need to be position independent when doing shared and pie links. The attached patch fixes the failing link command although I'm not sure it's optimal as generally -fPIE and -pie need to be specified together. Meson appears to test the options separately. Putting them together as one option doesn't work. The systemd build completes with the patch. There was one timeout in the testsuite but the system was under high load. Looking at the "ld" man page, it doesn't say anything about -fPIE, only about -pie. Can you please provide some references why on hppa the linker needs -fPIE in addition to -pie -fPIE is not a "ld" option. It is a compiler option. Note that the code uses the "cc", compiler driver, to perform the link. The "-flto" option also is a compiler driver option that enables link-time optimization. Take a look at what happens when you do a shared or pie link with compiler driver. Add "-v" to see the intermediate steps. -fPIC Generate position-independent code if possible (large mode). -fPIE Generate position-independent code for executables if possible (large mode). -flto Enable link-time optimization. -pie Create a dynamically linked position independent executable. -shared Create a shared library. Dave -- John David Anglin dave.ang...@bell.net
Bug#909396: systemd: FTBFS on hppa and x32 - relocation can not be used when making a shared object
Am 23.09.18 um 22:44 schrieb Michael Biebl: > Looking at the "ld" man page, it doesn't say anything about -fPIE, only > about -pie. > Can you please provide some references why on hppa the linker needs > -fPIE in addition to -pie Fwiw, this command succeeded for 239-7 [425/1545] cc -o src/udev/ata_id 'src/udev/src@udev@@ata_id@exe/ata_id_ata_id.c.o' -flto -Wl,--no-undefined -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -pie -Wl,--gc-sections -g -O2 -fdebug-prefix-map=/<>=. -Wformat -Werror=format-security -Wl,--start-group src/udev/libudev_static.a src/shared/libsystemd-shared-239.a src/libsystemd/libsystemd_static.a src/basic/libbasic.a src/udev/libudev-basic.a -lrt /usr/lib/hppa-linux-gnu/libcap.so -lacl /usr/lib/hppa-linux-gnu/libcryptsetup.so -lgcrypt /usr/lib/hppa-linux-gnu/libip4tc.so /usr/lib/hppa-linux-gnu/libip6tc.so /usr/lib/hppa-linux-gnu/libselinux.so /usr/lib/hppa-linux-gnu/libidn.so /usr/lib/hppa-linux-gnu/liblzma.so /usr/lib/hppa-linux-gnu/liblz4.so /usr/lib/hppa-linux-gnu/libblkid.so -lrt /usr/lib/hppa-linux-gnu/libmount.so -Wl,--end-group Why is this now suddenly failing on hppa? -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? signature.asc Description: OpenPGP digital signature
Bug#909396: systemd: FTBFS on hppa and x32 - relocation can not be used when making a shared object
Am 23.09.18 um 22:27 schrieb John David Anglin: > Well the problem was latent in 239-7 and probably gcc changed. The > thing is the error points > to an error in the link command. On hppa, it is almost always necessary > to link shared and pie > objects with -fPIC or -fPIE, respectively. There are compilations in > the link process and > the generated objects need to be position independent when doing shared > and pie links. > > The attached patch fixes the failing link command although I'm not sure > it's optimal as > generally -fPIE and -pie need to be specified together. Meson appears > to test the options > separately. Putting them together as one option doesn't work. The > systemd build completes > with the patch. There was one timeout in the testsuite but the system > was under high load. Looking at the "ld" man page, it doesn't say anything about -fPIE, only about -pie. Can you please provide some references why on hppa the linker needs -fPIE in addition to -pie -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? signature.asc Description: OpenPGP digital signature
Bug#909396: systemd: FTBFS on hppa and x32 - relocation can not be used when making a shared object
On 2018-09-23 4:06 AM, Michael Biebl wrote: One obvious difference is that 239-7 was compiled with meson 0.47.1-1, 239-8 and 239-9 with 0.47.2-1 The systemd build breaks completely with meson 0.48.0-1. Dave -- John David Anglin dave.ang...@bell.net
Bug#909396: systemd: FTBFS on hppa and x32 - relocation can not be used when making a shared object
On 2018-09-23 9:27 AM, Michael Biebl wrote: Am 23.09.18 um 14:43 schrieb John David Anglin: On 2018-09-23 4:06 AM, Michael Biebl wrote: Am 23.09.18 um 10:01 schrieb Michael Biebl: There weren't any relevant changes between 239-7 and 239-9 afaics, which makes me think that this is rather a toolchain issue on hppa and not a bug in systemd itself. Would be great if you can investigate this. One obvious difference is that 239-7 was compiled with meson 0.47.1-1, 239-8 and 239-9 with 0.47.2-1 Okay. I don't know anything about systemd build and where one would modify link options. I can try if you can suggest a possible location for change. It might be the "-flto" option would change things. It causes various compilations to occur which might cause error linking. My point is, that the systemd build system did not change between 239-7 and 239-8. So I suspect the root cause is elsewhere and not in systemd directly. Well the problem was latent in 239-7 and probably gcc changed. The thing is the error points to an error in the link command. On hppa, it is almost always necessary to link shared and pie objects with -fPIC or -fPIE, respectively. There are compilations in the link process and the generated objects need to be position independent when doing shared and pie links. The attached patch fixes the failing link command although I'm not sure it's optimal as generally -fPIE and -pie need to be specified together. Meson appears to test the options separately. Putting them together as one option doesn't work. The systemd build completes with the patch. There was one timeout in the testsuite but the system was under high load. Dave -- John David Anglin dave.ang...@bell.net --- meson.build.save2018-09-23 14:27:45.765707025 -0400 +++ meson.build 2018-09-23 14:40:20.002886449 -0400 @@ -341,7 +341,7 @@ # enable it when we are linking against them if not fuzzer_build possible_cc_flags += '-fPIE' -possible_link_flags += '-pie' +possible_link_flags += [ '-fPIE', '-pie', ] endif if cc.get_id() == 'clang'
Bug#909396: systemd: FTBFS on hppa and x32 - relocation can not be used when making a shared object
Am 23.09.18 um 14:43 schrieb John David Anglin: > On 2018-09-23 4:06 AM, Michael Biebl wrote: >> Am 23.09.18 um 10:01 schrieb Michael Biebl: >> >>> There weren't any relevant changes between 239-7 and 239-9 afaics, which >>> makes me think that this is rather a toolchain issue on hppa and not a >>> bug in systemd itself. >>> >>> Would be great if you can investigate this. >> One obvious difference is that 239-7 was compiled with meson 0.47.1-1, >> 239-8 and 239-9 with 0.47.2-1 > Okay. I don't know anything about systemd build and where one would > modify link options. > I can try if you can suggest a possible location for change. > > It might be the "-flto" option would change things. It causes various > compilations to occur > which might cause error linking. My point is, that the systemd build system did not change between 239-7 and 239-8. So I suspect the root cause is elsewhere and not in systemd directly. -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? signature.asc Description: OpenPGP digital signature
Bug#909396: systemd: FTBFS on hppa and x32 - relocation can not be used when making a shared object
On 2018-09-23 4:06 AM, Michael Biebl wrote: Am 23.09.18 um 10:01 schrieb Michael Biebl: There weren't any relevant changes between 239-7 and 239-9 afaics, which makes me think that this is rather a toolchain issue on hppa and not a bug in systemd itself. Would be great if you can investigate this. One obvious difference is that 239-7 was compiled with meson 0.47.1-1, 239-8 and 239-9 with 0.47.2-1 Okay. I don't know anything about systemd build and where one would modify link options. I can try if you can suggest a possible location for change. It might be the "-flto" option would change things. It causes various compilations to occur which might cause error linking. -- John David Anglin dave.ang...@bell.net
Bug#909396: systemd: FTBFS on hppa and x32 - relocation can not be used when making a shared object
Am 23.09.18 um 10:01 schrieb Michael Biebl: > There weren't any relevant changes between 239-7 and 239-9 afaics, which > makes me think that this is rather a toolchain issue on hppa and not a > bug in systemd itself. > > Would be great if you can investigate this. One obvious difference is that 239-7 was compiled with meson 0.47.1-1, 239-8 and 239-9 with 0.47.2-1 -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? signature.asc Description: OpenPGP digital signature
Bug#909396: systemd: FTBFS on hppa and x32 - relocation can not be used when making a shared object
Control: tags -1 + moreinfo Am 23.09.18 um 09:48 schrieb Michael Biebl: > Control: tags -1 + help > > Am 23.09.18 um 00:45 schrieb John David Anglin: >> I believe the error message is correct. Please add "-fPIC" or "-fPIE" to the >> link command. > > Can you provide a tested patch? > I.e. one which let's the build and the test-suite pass. Looking at https://buildd.debian.org/status/logs.php?pkg=systemd=hppa , systemd built fine in the past, say 239-7 There weren't any relevant changes between 239-7 and 239-9 afaics, which makes me think that this is rather a toolchain issue on hppa and not a bug in systemd itself. Would be great if you can investigate this. Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? signature.asc Description: OpenPGP digital signature
Bug#909396: systemd: FTBFS on hppa and x32 - relocation can not be used when making a shared object
Control: tags -1 + help Am 23.09.18 um 00:45 schrieb John David Anglin: > Source: systemd > Version: 239-9 > Severity: normal > > Dear Maintainer, > > The build fails here: > > [416/1547] cc -o src/udev/ata_id > 'src/udev/src@udev@@ata_id@exe/ata_id_ata_id.c.o' -flto -Wl,--no-undefined > -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -pie -Wl,--gc-sections -g -O2 > -fdebug-prefix-map=/<>=. -Wformat -Werror=format-security > -Wl,--start-group src/udev/libudev_static.a > src/shared/libsystemd-shared-239.a src/libsystemd/libsystemd_static.a > src/basic/libbasic.a src/udev/libudev-basic.a -lrt > /usr/lib/hppa-linux-gnu/libcap.so -lacl > /usr/lib/hppa-linux-gnu/libcryptsetup.so -lgcrypt > /usr/lib/hppa-linux-gnu/libip4tc.so /usr/lib/hppa-linux-gnu/libip6tc.so > /usr/lib/hppa-linux-gnu/libselinux.so /usr/lib/hppa-linux-gnu/libidn.so > /usr/lib/hppa-linux-gnu/liblzma.so /usr/lib/hppa-linux-gnu/liblz4.so > /usr/lib/hppa-linux-gnu/libblkid.so -lrt /usr/lib/hppa-linux-gnu/libmount.so > -Wl,--end-group -pthread > FAILED: src/udev/ata_id > cc -o src/udev/ata_id 'src/udev/src@udev@@ata_id@exe/ata_id_ata_id.c.o' > -flto -Wl,--no-undefined -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -pie > -Wl,--gc-sections -g -O2 -fdebug-prefix-map=/<>=. -Wformat > -Werror=format-security -Wl,--start-group src/udev/libudev_static.a > src/shared/libsystemd-shared-239.a src/libsystemd/libsystemd_static.a > src/basic/libbasic.a src/udev/libudev-basic.a -lrt > /usr/lib/hppa-linux-gnu/libcap.so -lacl > /usr/lib/hppa-linux-gnu/libcryptsetup.so -lgcrypt > /usr/lib/hppa-linux-gnu/libip4tc.so /usr/lib/hppa-linux-gnu/libip6tc.so > /usr/lib/hppa-linux-gnu/libselinux.so /usr/lib/hppa-linux-gnu/libidn.so > /usr/lib/hppa-linux-gnu/liblzma.so /usr/lib/hppa-linux-gnu/liblz4.so > /usr/lib/hppa-linux-gnu/libblkid.so -lrt /usr/lib/hppa-linux-gnu/libmount.so > -Wl,--end-group -pthread > /usr/bin/ld: /tmp/ccHOKcGI.ltrans0.ltrans.o: relocation R_PARISC_DPREL21L can > not be used when making a shared object; recompile with -fPIC > /usr/bin/ld: BFD (GNU Binutils for Debian) 2.31.1 internal error, aborting at > ../../bfd/elf32-hppa.c:3956 in elf32_hppa_relocate_section > > /usr/bin/ld: Please report this bug. > > collect2: error: ld returned 1 exit status > > I believe the error message is correct. Please add "-fPIC" or "-fPIE" to the > link command. Can you provide a tested patch? I.e. one which let's the build and the test-suite pass. Thanks, Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? signature.asc Description: OpenPGP digital signature
