On 8/9/21 7:39 AM, Tony Battersby wrote:
On 8/6/21 10:19 PM, Khem Raj wrote:
I am seeing bunch of failures on meta-oe and meta-atmel
https://errors.yoctoproject.org/Errors/Details/600064/
https://errors.yoctoproject.org/Errors/Details/600065/
https://errors.yoctoproject.org/Errors/Details/600071/
I sent patches for meta-openembedded, but I am not sure how to build
meta-atmel:
bitbake libegl
ERROR: Layer atmel is not compatible with the core layer which only
supports these series: honister (layer is compatible with hardknott)
you can use my branch which has fixes for override problem
https://github.com/YoeDistro/meta-atmel/tree/yoe/mut
But the fix is probably the same as the ones I sent for the other
packages (which I copied from
meta-openembedded/meta-networking/recipes-protocols/net-snmp/net-snmp_5.9.1.bb).
Tony Battersby
On Fri, Aug 6, 2021 at 2:04 PM Tony Battersby <to...@cybernetics.com> wrote:
On 8/6/21 12:44 PM, Tony Battersby wrote:
On 8/6/21 1:33 AM, Richard Purdie wrote:
On Thu, 2021-08-05 at 17:54 -0400, Tony Battersby wrote:
On 8/5/21 6:21 AM, Richard Purdie wrote:
On Wed, 2021-08-04 at 20:13 -0400, Randy MacLeod wrote:
On 2021-07-27 9:34 a.m., Tony Battersby wrote:
Currently, enabling Link Time Optimization via:
require conf/distro/include/lto.inc
DISTRO_FEATURES_append = " lto"
makes the build non-reproducible (as in
https://reproducible-builds.org/). This patch set fixes some of the
problems, but is not yet a complete solution.
This patch set addresses the following bugs:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=14481
https://bugzilla.yoctoproject.org/show_bug.cgi?id=14490
openembedded-core patches
1/4: lto.inc: disable LTO for grub
2/4: gcc: Backport patch to make LTO builds more reproducible
3/4: bitbake.conf: add DEBUG_PREFIX_MAP to TARGET_LDFLAGS
openembedded-devel patches
4/4: net-snmp: fix QA Issue after LDFLAGS change
Patch 3 adds DEBUG_PREFIX_MAP to TARGET_LDFLAGS (it was already in
TARGET_CFLAGS). This could possibly be done differently, such as by
removing it from TARGET_CFLAGS and adding it to TOOLCHAIN_OPTIONS,
TARGET_CC_ARCH, or TUNE_CCARGS, but I haven't tested those. Let me know
if there is a better/preferred way to do it.
The TARGET_LDFLAGS change broke net-snmp from meta-networking, so I have
included a patch to fix that. Other packages might have been broken as
well, although I haven't found any others.
With all these patches, there are still many shared libraries in /lib
and /usr/lib that are still not reproducible with LTO enabled because
x86_64-poky-linux-libtool filters out the -f*-prefix-map flags added by
patch 3 when linking shared libraries. /usr/lib/libbz2.so.1.0.6 from
the libbz2 package from the bzip2 recipe is a good example. I do not
have a patch for that problem. I will leave that to someone else.
So this is what we need to achieve reproducible LTO builds:
*) This patch set
*) A fix for libtool
*) Add LTO to reproducible builds testing if appropriate (e.g.
meta/lib/oeqa/selftest/cases/reproducible.py, CI on Autobuilder)
*) Fixes for individual packages, if needed
*) Once everything works, update the documentation again (I previously
requested a documentation change for this bug, which will eventually
show up at
http://docs.yoctoproject.org/test-manual/reproducible-builds.html).
This is certainly a few steps in the right direction with a good outline
of what's next.
I suspect that Richard has been busy with other things like overrides
and the update to glibc-2.34.
Do you have any suggestions or questions about what we should do
with this patch series Richard?
Thanks for the reminder, I've been meaning to queue/test this which I will
do. The gcc version has been bumped but I ported the patch forward to it
and the patches are in master-next for testing. It is great to see this
support moving forward.
Much as it pains me to admit having any libtool knowledge, I may be able to
help with that bit. There is an untested patch below which may address that
issue, or at least give a good hint on how to do it if it doesn't work. I
don't have an environment to test easily right now.
Cheers,
Richard
I tested your libtool patch in master-next with the bzip2 recipe. It
works as intended; libbz2.so.1.0.6 and all the other output files are
now reproducible with LTO enabled. I also verified that libtool is
passing the -f*-prefix-map flags to the link command in log.do_compile
as intended.
I also verified that core-image-minimal builds successfully with the
libtool patch.
Thanks for testing that, sounds like it helps that issue. I'll queue that patch.
Unfortunately even with the ruby fix I queued, we still see reproduciblity
issues with the LDFLAGS change in curl-dev, gettext-ptest, ruby and ruby-dbg
packages:
https://autobuilder.yoctoproject.org/typhoon/#/builders/115/builds/515/steps/12/logs/stdio
We'll need to resolve those before I can merge the flags change.
Diffoscope output:
https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20210805-8jfd55ki/packages/diff-html/
Basically LDFLAGS get encoded into a couple of scripts and the ruby binary.
Cheers,
Richard
It looks like the ruby problem is being caused by MJIT_DLDFLAGS in
build/tmp/work/core2-64-poky-linux/ruby/3.0.2-r0/build/mjit_config.h.
There is already a patch
poky/meta/recipes-devtools/ruby/ruby/0001-template-Makefile.in-do-not-write-host-cross-cc-item.patch
that removes some of these items for the sake of reproducibiltiy. But I
do not know ruby, so I do not know the implications of doing the same
thing for MJIT_DLDFLAGS.
Tony Battersby
Richard,
I noticed that you already fixed the problems in curl and gettext in
master-next. Here is a patch for ruby if you haven't fixed that already.
Tony Battersby
From 9cfb9777ff6ef95a27ed6a03719c807365d0f116 Mon Sep 17 00:00:00 2001
From: Tony Battersby <to...@cybernetics.com>
Date: Fri, 6 Aug 2021 16:47:14 -0400
Subject: [PATCH] ruby: Fix reproducibility issue with LDFLAGS
If we add DEBUG_PREFIX_MAP into LDFLAGS, ruby and ruby-dbg are no
longer reproducible. Fix this.
Signed-off-by: Tony Battersby <to...@cybernetics.com>
---
...-Makefile.in-filter-out-f-prefix-map.patch | 42 +++++++++++++++++++
meta/recipes-devtools/ruby/ruby_3.0.2.bb | 1 +
2 files changed, 43 insertions(+)
create mode 100644
meta/recipes-devtools/ruby/ruby/0002-template-Makefile.in-filter-out-f-prefix-map.patch
diff --git
a/meta/recipes-devtools/ruby/ruby/0002-template-Makefile.in-filter-out-f-prefix-map.patch
b/meta/recipes-devtools/ruby/ruby/0002-template-Makefile.in-filter-out-f-prefix-map.patch
new file mode 100644
index 0000000000..9387506c26
--- /dev/null
+++
b/meta/recipes-devtools/ruby/ruby/0002-template-Makefile.in-filter-out-f-prefix-map.patch
@@ -0,0 +1,42 @@
+Subject: [PATCH] template/Makefile.in: filter out -f*prefix-map
+
+If we add DEBUG_PREFIX_MAP into LDFLAGS, ruby and ruby-dbg are no longer
+reproducible. Fix this.
+
+Upstream-Status: Inapproppriate [oe-core specific]
+Signed-off-by: Tony Battersby <to...@cybernetics.com>
+---
+--- a/tool/mjit_archflag.sh
++++ b/tool/mjit_archflag.sh
+@@ -7,6 +7,20 @@ quote() {
+ echo
+ }
+
++quote_filtered() {
++ printf "#${indent}define $1"
++ while shift && [ "$#" -gt 0 ]; do
++ case "$1" in
++ -ffile-prefix-map=*|-fdebug-prefix-map=*|-fmacro-prefix-map=*)
++ ;;
++ *)
++ printf ' "%s"'$sep "$1"
++ ;;
++ esac
++ done
++ echo
++}
++
+ archs=""
+ arch_flag=""
+
+--- a/template/Makefile.in
++++ b/template/Makefile.in
+@@ -666,7 +666,7 @@ mjit_config.h:
+ quote "MJIT_OPTFLAGS " $(MJIT_OPTFLAGS); \
+ quote "MJIT_DEBUGFLAGS " $(MJIT_DEBUGFLAGS); \
+ quote "MJIT_LDSHARED " ; \
+- quote "MJIT_DLDFLAGS MJIT_ARCHFLAG" $(MJIT_DLDFLAGS); \
++ quote_filtered "MJIT_DLDFLAGS MJIT_ARCHFLAG" $(MJIT_DLDFLAGS); \
+ quote "MJIT_LIBS " $(LIBRUBYARG_SHARED); \
+ quote 'PRELOADENV "@PRELOADENV@"'; \
+ indent=$${archs:+' '}; \
diff --git a/meta/recipes-devtools/ruby/ruby_3.0.2.bb
b/meta/recipes-devtools/ruby/ruby_3.0.2.bb
index 0b9c342def..38e594a59e 100644
--- a/meta/recipes-devtools/ruby/ruby_3.0.2.bb
+++ b/meta/recipes-devtools/ruby/ruby_3.0.2.bb
@@ -6,6 +6,7 @@ SRC_URI += " \
file://remove_has_include_macros.patch \
file://run-ptest \
file://0001-template-Makefile.in-do-not-write-host-cross-cc-item.patch \
+ file://0002-template-Makefile.in-filter-out-f-prefix-map.patch \
"
SRC_URI[sha256sum] =
"5085dee0ad9f06996a8acec7ebea4a8735e6fac22f22e2d98c3f2bc3bef7e6f1"
--
2.25.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154654):
https://lists.openembedded.org/g/openembedded-core/message/154654
Mute This Topic: https://lists.openembedded.org/mt/84481652/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
