Re: I disabled lto, but it comes back in via -config --libs
Hi, On Fri, Apr 29, 2022 at 10:50 PM Steve Langasek wrote: > I think your main question has been addressed. But I want to ask, why is it a > problem to have part of this code with LTO enabled? Generally we disable > LTO only when it is causing build failures. Was that the case here? I don't think it's related to *part* of the build using LTO, and the rest not. That was just an observation I made when I saw that krb5-config --libs was also exporting lto flags, and I filed [4] for it. What prompted disabling lto is that mariadb was crashing[1] when it couldn't initialize the io_uring queue due to a low RLIMIT_MEMLOCK value we have by default. That baffled upstream, as the code should be treating that exception[5], but instead we get an abort. They investigated[2] and suspected lto was to blame. I tried a build without lto, and it worked. Two things out of this, at least: - why wasn't the exception caught when lto was enabled? gcc bug, mariadb bug, or something else? - should we bump our RLIMIT_MEMLOCK value from 64kbytes to something higher? There is an upstream commit[3] in systemd to bump it to 8Mb. I added a systemd task to the bug[1] about this 1. https://launchpad.net/bugs/1970634 2. https://jira.mariadb.org/browse/MDEV-25633 3. https://github.com/systemd/systemd/commit/852b62507b2 4. https://bugs.launchpad.net/ubuntu/+source/krb5/+bug/1970979 5. https://bugs.launchpad.net/ubuntu/+source/mariadb-10.6/+bug/1970634/comments/7 -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: I disabled lto, but it comes back in via -config --libs
Hi Andreas, On Fri, Apr 29, 2022 at 10:50:21AM -0300, Andreas Hasenack wrote: > I disabled lto in a build according to the instructions from [1]: > export DEB_BUILD_MAINT_OPTIONS=optimize=-lto > But I saw that it was still present in some steps of the build. > Notably when krb5/gssapi was used: [...] > That sounds bad. It means only portions of the build will have lto > disabled, while others will flip it back on. How to sort this out? > Looks like the krb5 package needs some fixing, but I'm unsure how. I think your main question has been addressed. But I want to ask, why is it a problem to have part of this code with LTO enabled? Generally we disable LTO only when it is causing build failures. Was that the case here? -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developer https://www.debian.org/ slanga...@ubuntu.com vor...@debian.org signature.asc Description: PGP signature -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: I disabled lto, but it comes back in via -config --libs
Sergio Durigan Junior schrieb am Fr., 29. Apr. 2022, 20:14: > On Friday, April 29 2022, Andreas Hasenack wrote: > > > Hi, > > > > I disabled lto in a build according to the instructions from [1]: > > > > export DEB_BUILD_MAINT_OPTIONS=optimize=-lto > > > > But I saw that it was still present in some steps of the build. > > Notably when krb5/gssapi was used: > > ... > > -- Found GSSAPI: -L/usr/lib/x86_64-linux-gnu/mit-krb5 > > -Wl,-Bsymbolic-functions -flto=auto -ffat-lto-objects -flto=auto > > -Wl,-z,relro -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err > > ... > > /usr/bin/cc -fPIC -g -O2 > > -ffile-prefix-map=/home/ubuntu/git/packages/mariadb/mariadb-10.6=. > > -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time > > -D_FORTIFY_SOURCE=2 -Wdate-time -D_FORTIFY_SOURCE=2 -pie -fPIC > > -fstack-protector --param=ssp-buffer-size=4 -Wunused -Wlogical-op > > -Wno-uninitialized -Wall -Wextra -Wformat-security -Wno-init-self > > -Wwrite-strings -Wshift-count-overflow -Wdeclaration-after-statement > > -Wno-undef -Wno-unknown-pragmas -O2 -g -static-libgcc > > -fno-omit-frame-pointer -fno-strict-aliasing -Wno-uninitialized > > -fno-omit-frame-pointer -D_FORTIFY_SOURCE=2 -DDBUG_OFF -Wall > > -Wdeclaration-after-statement -Wextra -Wformat-security > > -Wno-format-truncation -Wno-init-self -Wno-nonnull-compare > > -Wno-unused-parameter -Wvla -Wwrite-strings -DDBUG_OFF > > -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,relro,-z,now -shared -o > > auth_gssapi_client.so > > CMakeFiles/auth_gssapi_client.dir/plugins/auth/auth_gssapi_client.c.o > > CMakeFiles/auth_gssapi_client.dir/plugins/auth/gssapi_client.c.o > > CMakeFiles/auth_gssapi_client.dir/plugins/auth/gssapi_errmsg.c.o > > -L/usr/lib/x86_64-linux-gnu/mit-krb5 -Wl,-Bsymbolic-functions > > -flto=auto -ffat-lto-objects -flto=auto -Wl,-z,relro -lgssapi_krb5 > > -lkrb5 -lk5crypto -lcom_err > > > > Indeed in this case it comes from kerberos/gssapi: > > > > $ krb5-config --libs gssapi > > -L/usr/lib/x86_64-linux-gnu/mit-krb5 -Wl,-Bsymbolic-functions > > -flto=auto -ffat-lto-objects -flto=auto -Wl,-z,relro -lgssapi_krb5 > > -lkrb5 -lk5crypto -lcom_err > > > > That sounds bad. It means only portions of the build will have lto > > disabled, while others will flip it back on. How to sort this out? > > Looks like the krb5 package needs some fixing, but I'm unsure how. > > Just reiterating what I told you during standup, this leakage is wrong > and should be fixed in krb5's pkg-config definitions. Wondering if it would make sense to have pkg-config (and pkgconf) strip stuff when reading the files, or do some fixup in debhelper that works everywhere rather than fixing this per package. But I also don't know how it got there. (sorry if phone messed up formatting) -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: I disabled lto, but it comes back in via -config --libs
On Friday, April 29 2022, Andreas Hasenack wrote: > Hi Sergio, thanks for the reply > > On Fri, Apr 29, 2022 at 3:14 PM Sergio Durigan Junior > wrote: >> Either way, I believe this issue should be addressed in krb5 as I said >> above. I haven't been able to find any bug about this in upstream's bug >> tracker. > > I filed https://bugs.launchpad.net/ubuntu/+source/krb5/+bug/1970979 > and also subscribed and emailed the upstream krbdev@ mailing list, but > I think it may be subscription-moderated. If my message doesn't show > up over the weekend, I'll file a bug in their bugtracker. Thanks, Andreas. I've subscribed myself to the bug, let's see how it unfolds. -- Sergio GPG key ID: E92F D0B3 6B14 F1F4 D8E0 EB2F 106D A1C8 C3CB BF14 -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: I disabled lto, but it comes back in via -config --libs
Hi Sergio, thanks for the reply On Fri, Apr 29, 2022 at 3:14 PM Sergio Durigan Junior wrote: > Either way, I believe this issue should be addressed in krb5 as I said > above. I haven't been able to find any bug about this in upstream's bug > tracker. I filed https://bugs.launchpad.net/ubuntu/+source/krb5/+bug/1970979 and also subscribed and emailed the upstream krbdev@ mailing list, but I think it may be subscription-moderated. If my message doesn't show up over the weekend, I'll file a bug in their bugtracker. -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: I disabled lto, but it comes back in via -config --libs
On Friday, April 29 2022, Andreas Hasenack wrote: > Hi, > > I disabled lto in a build according to the instructions from [1]: > > export DEB_BUILD_MAINT_OPTIONS=optimize=-lto > > But I saw that it was still present in some steps of the build. > Notably when krb5/gssapi was used: > ... > -- Found GSSAPI: -L/usr/lib/x86_64-linux-gnu/mit-krb5 > -Wl,-Bsymbolic-functions -flto=auto -ffat-lto-objects -flto=auto > -Wl,-z,relro -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err > ... > /usr/bin/cc -fPIC -g -O2 > -ffile-prefix-map=/home/ubuntu/git/packages/mariadb/mariadb-10.6=. > -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time > -D_FORTIFY_SOURCE=2 -Wdate-time -D_FORTIFY_SOURCE=2 -pie -fPIC > -fstack-protector --param=ssp-buffer-size=4 -Wunused -Wlogical-op > -Wno-uninitialized -Wall -Wextra -Wformat-security -Wno-init-self > -Wwrite-strings -Wshift-count-overflow -Wdeclaration-after-statement > -Wno-undef -Wno-unknown-pragmas -O2 -g -static-libgcc > -fno-omit-frame-pointer -fno-strict-aliasing -Wno-uninitialized > -fno-omit-frame-pointer -D_FORTIFY_SOURCE=2 -DDBUG_OFF -Wall > -Wdeclaration-after-statement -Wextra -Wformat-security > -Wno-format-truncation -Wno-init-self -Wno-nonnull-compare > -Wno-unused-parameter -Wvla -Wwrite-strings -DDBUG_OFF > -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,relro,-z,now -shared -o > auth_gssapi_client.so > CMakeFiles/auth_gssapi_client.dir/plugins/auth/auth_gssapi_client.c.o > CMakeFiles/auth_gssapi_client.dir/plugins/auth/gssapi_client.c.o > CMakeFiles/auth_gssapi_client.dir/plugins/auth/gssapi_errmsg.c.o > -L/usr/lib/x86_64-linux-gnu/mit-krb5 -Wl,-Bsymbolic-functions > -flto=auto -ffat-lto-objects -flto=auto -Wl,-z,relro -lgssapi_krb5 > -lkrb5 -lk5crypto -lcom_err > > Indeed in this case it comes from kerberos/gssapi: > > $ krb5-config --libs gssapi > -L/usr/lib/x86_64-linux-gnu/mit-krb5 -Wl,-Bsymbolic-functions > -flto=auto -ffat-lto-objects -flto=auto -Wl,-z,relro -lgssapi_krb5 > -lkrb5 -lk5crypto -lcom_err > > That sounds bad. It means only portions of the build will have lto > disabled, while others will flip it back on. How to sort this out? > Looks like the krb5 package needs some fixing, but I'm unsure how. Just reiterating what I told you during standup, this leakage is wrong and should be fixed in krb5's pkg-config definitions. Some interesting things to consider here: - Debian/Ubuntu differ from Fedora in the sense that we specify the LTO flags during link time as well (LDFLAGS), while Fedora only specifies them during compile time. GCC can auto-detect LTO bytecode during link time and will do the right thing even if no LTO flags are passed (as long as we use GCC to perform the link, which we do). So in theory we could remove the "-flto -ffat-lto-objects" from our LDFLAGS and still have LTO-enabled binaries. - As a direct consequence of the above, the exact same command yields a different result in Fedora, even though they also have LTO enabled. - GCC will also be smart enough when mix & matching object files and libraries that were compiled with and without LTO, so no worries here. Either way, I believe this issue should be addressed in krb5 as I said above. I haven't been able to find any bug about this in upstream's bug tracker. Thanks, -- Sergio GPG key ID: E92F D0B3 6B14 F1F4 D8E0 EB2F 106D A1C8 C3CB BF14 -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: I disabled lto, but it comes back in via -config --libs
Same thing with -Wl,-Bsymbolic-functions: https://lists.ubuntu.com/archives/ubuntu-devel/2018-June/040371.html On Fri, Apr 29, 2022 at 10:50 AM Andreas Hasenack wrote: > > Hi, > > I disabled lto in a build according to the instructions from [1]: > > export DEB_BUILD_MAINT_OPTIONS=optimize=-lto > > But I saw that it was still present in some steps of the build. > Notably when krb5/gssapi was used: > ... > -- Found GSSAPI: -L/usr/lib/x86_64-linux-gnu/mit-krb5 > -Wl,-Bsymbolic-functions -flto=auto -ffat-lto-objects -flto=auto > -Wl,-z,relro -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err > ... > /usr/bin/cc -fPIC -g -O2 > -ffile-prefix-map=/home/ubuntu/git/packages/mariadb/mariadb-10.6=. > -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time > -D_FORTIFY_SOURCE=2 -Wdate-time -D_FORTIFY_SOURCE=2 -pie -fPIC > -fstack-protector --param=ssp-buffer-size=4 -Wunused -Wlogical-op > -Wno-uninitialized -Wall -Wextra -Wformat-security -Wno-init-self > -Wwrite-strings -Wshift-count-overflow -Wdeclaration-after-statement > -Wno-undef -Wno-unknown-pragmas -O2 -g -static-libgcc > -fno-omit-frame-pointer -fno-strict-aliasing -Wno-uninitialized > -fno-omit-frame-pointer -D_FORTIFY_SOURCE=2 -DDBUG_OFF -Wall > -Wdeclaration-after-statement -Wextra -Wformat-security > -Wno-format-truncation -Wno-init-self -Wno-nonnull-compare > -Wno-unused-parameter -Wvla -Wwrite-strings -DDBUG_OFF > -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,relro,-z,now -shared -o > auth_gssapi_client.so > CMakeFiles/auth_gssapi_client.dir/plugins/auth/auth_gssapi_client.c.o > CMakeFiles/auth_gssapi_client.dir/plugins/auth/gssapi_client.c.o > CMakeFiles/auth_gssapi_client.dir/plugins/auth/gssapi_errmsg.c.o > -L/usr/lib/x86_64-linux-gnu/mit-krb5 -Wl,-Bsymbolic-functions > -flto=auto -ffat-lto-objects -flto=auto -Wl,-z,relro -lgssapi_krb5 > -lkrb5 -lk5crypto -lcom_err > > Indeed in this case it comes from kerberos/gssapi: > > $ krb5-config --libs gssapi > -L/usr/lib/x86_64-linux-gnu/mit-krb5 -Wl,-Bsymbolic-functions > -flto=auto -ffat-lto-objects -flto=auto -Wl,-z,relro -lgssapi_krb5 > -lkrb5 -lk5crypto -lcom_err > > That sounds bad. It means only portions of the build will have lto > disabled, while others will flip it back on. How to sort this out? > Looks like the krb5 package needs some fixing, but I'm unsure how. > > > 1. https://wiki.ubuntu.com/ToolChain/LTO -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
I disabled lto, but it comes back in via -config --libs
Hi, I disabled lto in a build according to the instructions from [1]: export DEB_BUILD_MAINT_OPTIONS=optimize=-lto But I saw that it was still present in some steps of the build. Notably when krb5/gssapi was used: ... -- Found GSSAPI: -L/usr/lib/x86_64-linux-gnu/mit-krb5 -Wl,-Bsymbolic-functions -flto=auto -ffat-lto-objects -flto=auto -Wl,-z,relro -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err ... /usr/bin/cc -fPIC -g -O2 -ffile-prefix-map=/home/ubuntu/git/packages/mariadb/mariadb-10.6=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -Wdate-time -D_FORTIFY_SOURCE=2 -pie -fPIC -fstack-protector --param=ssp-buffer-size=4 -Wunused -Wlogical-op -Wno-uninitialized -Wall -Wextra -Wformat-security -Wno-init-self -Wwrite-strings -Wshift-count-overflow -Wdeclaration-after-statement -Wno-undef -Wno-unknown-pragmas -O2 -g -static-libgcc -fno-omit-frame-pointer -fno-strict-aliasing -Wno-uninitialized -fno-omit-frame-pointer -D_FORTIFY_SOURCE=2 -DDBUG_OFF -Wall -Wdeclaration-after-statement -Wextra -Wformat-security -Wno-format-truncation -Wno-init-self -Wno-nonnull-compare -Wno-unused-parameter -Wvla -Wwrite-strings -DDBUG_OFF -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,relro,-z,now -shared -o auth_gssapi_client.so CMakeFiles/auth_gssapi_client.dir/plugins/auth/auth_gssapi_client.c.o CMakeFiles/auth_gssapi_client.dir/plugins/auth/gssapi_client.c.o CMakeFiles/auth_gssapi_client.dir/plugins/auth/gssapi_errmsg.c.o -L/usr/lib/x86_64-linux-gnu/mit-krb5 -Wl,-Bsymbolic-functions -flto=auto -ffat-lto-objects -flto=auto -Wl,-z,relro -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err Indeed in this case it comes from kerberos/gssapi: $ krb5-config --libs gssapi -L/usr/lib/x86_64-linux-gnu/mit-krb5 -Wl,-Bsymbolic-functions -flto=auto -ffat-lto-objects -flto=auto -Wl,-z,relro -lgssapi_krb5 -lkrb5 -lk5crypto -lcom_err That sounds bad. It means only portions of the build will have lto disabled, while others will flip it back on. How to sort this out? Looks like the krb5 package needs some fixing, but I'm unsure how. 1. https://wiki.ubuntu.com/ToolChain/LTO -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
