Re: Which conferences are folks attending these days?

2024-04-23 Thread Bernhard M. Wiedemann via rb-general

On 18/04/2024 15.45, Chris Lamb wrote:


To that end, what conferences are folks on this list still  going  to,
and, hopefully, still getting something from?  I mean, there  must  be
some exceptions other than FOSDEM… :)


My list has become rather short:
rb conf (if within Europe)
openSUSE conf, Nuremberg
and a mini-openSUSE conf in Berlin, co-located with SUSECon


OBS/rpm & java-21 success

2024-03-31 Thread Bernhard M. Wiedemann via rb-general

Hi,

today I want to share with you two successes on our path to total 
reproducibility in openSUSE:


Through the persistence of my colleague Jan Zerebecki and the help of 
mls (SUSE's rpm maintainer) we made nice progress on

https://bugzilla.opensuse.org/show_bug.cgi?id=1148824
to finally normalize mtimes in official openSUSE Tumbleweed rpms.

Together with a workaround for
https://github.com/rpm-software-management/rpm/issues/2965
this allowed me to create bit-identical rpms to the ones pulled from 
build.opensuse.org , processed with rpm --delsign


Now everything that was reproducible in my QA-tests is also 
reproducible+verifiable in practice.



The other success is that I saw 2 bit-identical java-21-openjdk rpm 
builds, but only when both were done on 1-core VMs, so there might only 
be some raciness left. [1]

javadoc output still has an issue from filesystem-readdir-order.
We have a build-tool workaround for that in place [2]


Ciao
Bernhard M.


[1] 
https://rb.zq1.de/compare.factory-20240331/diffs/java-21-openjdk-compare.out
[2] 
https://github.com/bmwiedemann/openSUSE/blob/54e27e1/packages/_/_project/_config#L19-L20


OpenPGP_signature.asc
Description: OpenPGP digital signature


Re: Arch Linux minimal container userland 100% reproducible - now what?

2024-03-24 Thread Bernhard M. Wiedemann via rb-general

On 21/03/2024 21.38, kpcyrd wrote:
- libjpeg-turbo: this package contains a .jar file that is built by 
CMake and contains timestamps of the buildtime, but there's no way in 
CMake to pass --date to the jar executable to normalize this


You could use strip-nondeterminism for post-processing there.
For some reason it is reproducible in my openSUSE tests without us doing 
any extra steps.

https://ismypackagereproducibleyet.org/?pkg=libjpeg-turbo


- librsvg: the 3 rebuilders I've checked produced a .text section that is 6 bytes shorter (0x2dda2c vs 0x2dda26), I didn't investigate further yet, the diff is quite long because a lot of addresses are mismatching as a consequence 


My notes have https://gitlab.gnome.org/GNOME/librsvg/-/issues/1015 which 
turned out to be from pango mis-rendering text when font files were absent.


Ciao
Bernhard M.


Re: Why is not everything reproducible yet?

2024-02-14 Thread Bernhard M. Wiedemann via rb-general



On 14/02/2024 16.19, Santiago Torres-Arias wrote:

1. can we study the conflicting interestes (i.e., above) that stop
reproducibility from happening.


Yes, that should be possible. The above summarized my experience from 
the 1000 patches and bug-reports I did and the interactions with various 
upstreams.

The links are public and recorded in the monthly reports
https://salsa.debian.org/reproducible-builds/reproducible-website/-/tree/master/_reports
and earlier weekly posts
https://salsa.debian.org/reproducible-builds/reproducible-website/-/tree/master/_blog/posts

I can probably provide more input for such a study.


2. Are misunderstandings about reproducibility getting in the way from
pushing to it (e.g., the notion that docker containers are
inherrently reproducible). Is the perfect the enemy of the good?
what notions of reproducibility exist and how can we build a
roadmap from the weak to the strong?


There are some.
One is the confusion with what we started to call "repeatable builds" = 
the ability to be able to do a second build with the same explicit 
inputs. SBOMs help with repeatable builds, but if they become embedded 
in the build output, they can even hinder some side-benefits of 
reproducible builds, because every minor change in inputs now causes a 
change in output.


The other thing was
https://web.archive.org/web/20200807033032/https://blog.cmpxchg8b.com/2020/07/you-dont-need-reproducible-builds.html
that gained some anti-r-b mindshare, even though it neglegted several 
important aspects. E.g. it mentions the risk of stealing source-code 
which obviously does not apply to FLOSS.




3. What other uses of r-b exist beyond the malicious toolchain example?
can we use them as leverage to increase interest in the space?


On a past r-b summit we collected
https://reproducible-builds.org/docs/buy-in/
e.g. in openSUSE we always pushed for some level of binary equivalence 
to do build-tree-pruning in our open-build-service to save build power, 
shorten rebuild time and save bandwidth for mirrors and users that do 
not need to update unchanged packages.
We also publish updates as delta-rpm-packages that probably were more 
compact with fewer random variations.


The page also lists the QA aspect. I did find a dozen corruption bugs 
that went unnoticed for years.


e.g. https://gitlab.gnome.org/GNOME/libxslt/-/issues/37 had this 
memorable quote from upstream:
This was caused by an interesting bug in libxml2's streaming XPath engine. I'm still puzzled why it took so long to discover this issue. 


So for your study, you could find this link in _reports/2020-04.md

another corruption bug in
_reports/2023-10.md:* 
[`OpenRGB`](https://gitlab.com/CalcProgrammer1/OpenRGB/-/issues/3675) 
([corruption-related 
issue](https://gitlab.com/CalcProgrammer1/OpenRGB/-/merge_requests/2103))



One benefit not listed is that with r-b it is possible to say "version 
1.2.3 has hash abcdef" and you can provide a signature of the file, 
without uploading the file itself. With content-addressable storage such 
as IPFS, you can then also link to such an artifact and anyone else can 
provide the correct file.


e.g. in
http://bafybeiezodttpdsrhy7gj7zuzklbs3exh42a4ezorsepnn74ar2gkicujy.ipfs.cf-ipfs.com/
if we had reproducible ISOs, I could build and sign them in a 
low-bandwidth place but build+upload from another.




Ciao
Bernhard M.


OpenPGP_signature.asc
Description: OpenPGP digital signature


Re: Potential issues with the snippet to parse SOURCE_DATE_EPOCH in C

2024-01-21 Thread Bernhard M. Wiedemann via rb-general



On 19/01/2024 21.03, Chris Lamb wrote:

Was there any reason to reject >ULONG_MAX? I'm touching this code,
and don't see a reason for it; it looks very arbitrary; especially
since some systems can have 32-bit long, but 64-bit time_t. Should I
just drop that check, or keep it? And why?


There is another issue with the ULONG_MAX - that is it allows to 
represent timestamps up to 2106, but a 32-bit time_t is a signed int32, 
so will roll over in 2038 back to 1901.


At some (not too far) point in time, programs and libraries compiling 
with glibc should start to build with -D_TIME_BITS=64 
-D_FILE_OFFSET_BITS=64 to get 64-bit time_t everywhere. [1]


Ciao
Bernhard M.

[1] 
https://www.reddit.com/r/linux/comments/19a95cl/today_is_y2k38_commemoration_day_t14/


OpenPGP_signature.asc
Description: OpenPGP digital signature


Why is not everything reproducible yet?

2023-12-20 Thread Bernhard M. Wiedemann via rb-general

Sometimes people wonder:
Why is not everything reproducible yet?

And the general reason is that there are other interests that result in 
added non-determinism.

I collected some with examples



Performance (PGO, benchmarking, -march=native, parallelism/races)
 https://build.opensuse.org/request/show/1130552
 https://github.com/bmwiedemann/theunreproduciblepackage/tree/master/pgo


Simplicity (e.g. using random UUIDs instead of hashed inputs)
 https://github.com/ipxe/ipxe/pull/1082


Security (Signatures):
 https://bugzilla.opensuse.org/show_bug.cgi?id=1217690
 https://bugzilla.opensuse.org/show_bug.cgi?id=1208478
 https://bugzilla.opensuse.org/show_bug.cgi?id=1081723


Traceability of provenance (date+user+hostname):

https://github.com/bmwiedemann/theunreproduciblepackage/tree/master/timestamp


repeatable builds:
 https://github.com/rpm-software-management/rpm/issues/2343


Portability:
 https://github.com/ipxe/ipxe/pull/1082#issuecomment-1862899660
 - see also the code-monster we need to support SOURCE_DATE_EPOCH with 
sh on *NIX



Ciao
Bernhard M.


OpenPGP_signature.asc
Description: OpenPGP digital signature


Re: scheme and lisp

2023-11-23 Thread Bernhard M. Wiedemann via rb-general



On 23/11/2023 17.53, Ludovic Courtès wrote:

The implementations are also very different: for instance, Chez
implements a native ahead-of-time compiler whereas Guile has bytecode
compilation plus just-in-time compilation.  Thus problems and solutions
for one implementation are unlikely to translate to other
implementations.


Thanks for that insight.


That said I’m surprised about Emacs, this needs more investigation…


Here is the diff from our emacs-29.1
https://rb.zq1.de/compare.factory-20230830/diffs/emacs-compare.out
it has 2 different variations in .eln and .pdmp files.
The .eln one probably comes from ASLR [1].
While the .pdmp has some sequences of very random bytes.

Ciao
Bernhard M.

[1] https://github.com/bmwiedemann/theunreproduciblepackage/tree/master/aslr


OpenPGP_signature.asc
Description: OpenPGP digital signature


scheme and lisp

2023-11-23 Thread Bernhard M. Wiedemann via rb-general

Hi,

in openSUSE there are some packages that so far refuse to build 
reproducibly. The common theme around them is that they use scheme or 
lisp to produce binaries with a 'dump' command.


e.g. for scheme48 I extracted this reproducer:

pushd ~/rpmbuild/BUILD/scheme48-*/ps-compiler
../go -h 2000 -a batch <<- 'EOF'
,config ,load ../scheme/prescheme/interface.scm
,config ,load ../scheme/prescheme/package-defs.scm
,exec ,load load-ps-compiler.scm
,in prescheme-compiler prescheme-compiler
,user (define prescheme-compiler ##)
,dump ../ps-compiler.image "(Pre-Scheme)"
,exit
EOF

I also know that guile implements scheme and builds reproducibly (with 
-j1). So there must be a way to do it right.


The list of our packages I think are affected by this is:
clisp
scheme48
chezscheme
emacs
maxima
scsh
xindy

Most distros seem to be affected by this:
http://ismypackagereproducibleyet.org/?pkg=scheme48
http://ismypackagereproducibleyet.org/?pkg=emacs surprisingly shows as 
reproducible in Archlinux, but I could not figure out why.

maxima also shows as green there.

Can we get them reproducible? Or can we drop+replace these 
implementations with guile?


I'd appreciate some insight from this knowledgeable crowd.

Ciao
Bernhard M.


OpenPGP_signature.asc
Description: OpenPGP digital signature


Re: Reproducibility terminology/definitions

2023-11-11 Thread Bernhard M. Wiedemann via rb-general



On 08/11/2023 16.38, Pol Dellaiera wrote:

you define functions doing I/O as Impure functions.
But without I/O, no build output can be written, so all builds must use 
impure functions.



In practice we see non-determinism from approx 10 sources, such as 
documented in https://github.com/bmwiedemann/theunreproduciblepackage/


e.g.
https://github.com/bmwiedemann/theunreproduciblepackage/blob/master/race/race.2.sh

#!/bin/sh
racepart()
{
input=$1
sleep 0.1
echo $input
}
for i in $(seq 1 10) ; do
# & backgrounds the process to do parallel processing
racepart $i &
done


OpenPGP_signature.asc
Description: OpenPGP digital signature


LibreOffice success story

2023-11-07 Thread Bernhard M. Wiedemann via rb-general

Dear fellow R-B-ings

Just 2 weeks ago, when I re-reviewed the remaining ~120 major issues in 
openSUSE, I pretty much skipped over LibreOffice (and only this one), 
noting it down as "various issues", because some years ago, when I had 
previously taken a closer look, there had been so many issues in files 
of various formats and additionally the build of that large package took 
hours, so that I had quickly stopped debugging and left it "for later".


Then on the r-b-summit I met Thorsten from LibreOffice upstream and 
together we reviewed the old diff I had from 2022 [1]. We noticed that 
there was an issue with docs, but I had recently made

https://code.opensuse.org/package/doxygen/blob/master/f/reproducible.patch
that turned out to solve this exact issue. It still needs upstreaming.

There was a timestamp-issue from long-orphaned clucene and we found 3 
others that Thorsten quickly fixed:

https://gerrit.libreoffice.org/q/topic:reprobuild

Now there were only mtimes left in .jar and .zip files that were easily 
normalized with strip-nondeterminism.


So today I hold in my hands the first two bit-identical LibreOffice rpm 
packages.

And this is the success I wanted to share with you all today.

It makes me feel as if we can solve anything.

Ciao
Bernhard M.


[1] https://rb.zq1.de/compare.factory-20230109/diffs/libreoffice-compare.out


OpenPGP_signature.asc
Description: OpenPGP digital signature


Re: Bug#1051801: document DEB_BUILD_OPTIONS value nopgo

2023-09-13 Thread Bernhard M. Wiedemann via rb-general

On 11/09/2023 09.25, Helmut Grohne wrote:

It also
is unclear how it affects reproducible builds since such builds depend
on the performance characteristics of the system performing the build.


It is worth noting that the performance (execution time) of a 
build-system does not matter for profiling, so it is possible to achieve 
reproducible builds with PGO enabled. It is just hard.


See https://build.opensuse.org/request/show/499887
linked in 
https://github.com/bmwiedemann/theunreproduciblepackage/tree/master/pgo


In that gzip patch we even had to hide the name of the temporary file 
from gzip to not get variations from a 'tolower' call that would be 
optimized for different amounts of upper/lower-case letters.


Parallel builds and variations in ordering are also problematic, because 
some of the performance-counter-logic in gcc is not commutative, so 
running A and B calls produces different results from first calling B 
and then A.



With all that said, in openSUSE we also have a %do_profiling value to 
disable PGO for gcc, python and some others, because these profiling 
runs are too large to make deterministic.


Ciao
Bernhard M.


Re: Reproducible Arch Linux (August 2023)

2023-08-25 Thread Bernhard M. Wiedemann via rb-general

On 25/08/2023 06.56, kpcyrd wrote:


It seems the order for this has an impact on the elf binary.

```
find . -type f -perm -u+w -print0 2>/dev/null | while IFS= read -rd '' 
binary ; do


This should be trivial to fix with

find . -type f -perm -u+w -print0 2>/dev/null |
 sort -z |
 while IFS= read -rd '' binary ; do



For some reason, this did not show up in our
https://github.com/openSUSE/brp-check-suse/blob/master/brp-15-strip-debug#L22

but then I run most tests without debuginfo (for performance) and only 
use debuginfo in verification builds.



Ciao
Bernhard M.


Re: trying to reproduce hello-traditional from Debian. .buildinfo file? next steps?

2023-08-02 Thread Bernhard M. Wiedemann via rb-general

On 02/08/2023 11.26, Carles Pina i Estany wrote:


Hi,

This is Debian specific but I cannot find a reproducible builds Debian
specific mailing list. Let me know if I should ask elsewhere. Feel free
to send me some pointers to read it myself.

TL;DR: I'm trying to build hello-traditional from Debian and have the
same result as Debian. I cannot do it. Pointers welcome. I thought of
using the .buildinfo file to reproduce the build environment and deps
but unsure of the best way and if this is the way.

I'm trying to reproduce the build of the package hello-traditional. I
understand from here:
https://tests.reproducible-builds.org/debian/rb-pkg/bookworm/amd64/hello-traditional.html

That should be reproducible.

I've done:
$ sbuild --no-clean --arch-any --arch-all --no-source --dist=stable 
--arch=amd64 
http://deb.debian.org/debian/pool/main/h/hello-traditional/hello-traditional_2.10-6.dsc


https://www.reddit.com/r/reproduciblebuilds/comments/tqrf9q/the_binary_that_varies_from_full_moon/

It might be something else, but since you mentioned "hello", there is a 
chance that this amazing story is relevant.



Ciao
Bernhard M.


Re: Introducing: Semantically reproducible builds

2023-05-29 Thread Bernhard M. Wiedemann via rb-general



On 29/05/2023 06.10, Vagrant Cascadian wrote:

Do such tools actually exist, or are we talking about something
theoretical here?


https://github.com/openSUSE/build-compare/ is in use for 13 years.

And strip-nondeterminism can be used to build another such tool.

They will only ever be able to normalize or ignore certain known classes 
of differences. It is good enough to avoid review of many diffs.


e.g. https://rb.zq1.de/compare.factory/report-202303.txt has
not-bit-by-bit-identical: 673
build-compare-failed: 483

So for 190 packages build-compare found that they only had insignificant 
diffs and were considered semantically equivalent, so I could spend more 
time, debugging the other 483 diffs.



I very much worry that the meaning of Reproducible Builds may gradually
get whittled down


I share this concern, which is why I have been calling this 
semi-reproducible to distinguish it from bit-reproducible / 
fully-reproducible.
That 'semi-' prefix should give people a good hint of what it is and if 
not, encourage them to ask for details. "sort-of-reproducible" or 
"almost-but-not-quite-reproducible" could also be an option :-)



Ciao
Bernhard M.


OpenPGP_signature
Description: OpenPGP digital signature


Re: Introducing: Semantically reproducible builds

2023-05-29 Thread Bernhard M. Wiedemann via rb-general



On 29/05/2023 05.25, David A. Wheeler wrote:

If you have tips on common likely errors, please post, I think
that would be of interest to many.


https://github.com/openSUSE/build-compare/issues/53
https://github.com/openSUSE/build-compare/issues/33
https://github.com/openSUSE/build-compare/pull/36
https://github.com/openSUSE/build-compare/pull/28

We use bash there to not add dependencies.
Looking at the bugs, those were mostly problems of tracking state in 
variables.


It would be less troublesome if we would not use it like diffoscope to 
report all diffs, but instead exit on the first relevant diff to keep it 
simple.


The cleaner way is to use strip-nondeterminism to remove all these 
insignificant bits during build and make the resulting bit-reproducible 
output the official binary.


As a *recipient* who has no control over the build process used by
someone else to create their package, I need some workable
alternatives to estimate risk.


A recipient could still use strip-nondeterminism (and custom sed) on 
both files before calling diff.

Testing for bit-identity is trivial.
Testing for semantic equivalence is not.

To ensure that the filters did not remove significant parts (e.g. sed 
/.*//), they should then use the filtered version in production.



Ciao
Bernhard M.


OpenPGP_signature
Description: OpenPGP digital signature


Re: Introducing: Semantically reproducible builds

2023-05-28 Thread Bernhard M. Wiedemann via rb-general
I agree, that it is good to give it a name (I have called it 
semi-reproducible before), but we should be clear on communicating the 
disadvantages.


In openSUSE we have been working towards repeatable semantically 
reproducible builds for over a decade [1] using our open-build-service 
and a tool called build-compare to filter out "insignificant" diffs.


However, while working with the tool, I already found three (3) bugs in 
build-compare that made it report packages with significant differences 
as 'identical'.
And if you don't rely on such tools, you need expensive manual reviews 
every time that cannot be automated and might also miss issues.


I have manually reviewed hundreds of package diffs in the past and it 
took many hours, so I'm not eager to repeat that.



Another disadvantage of such binaries is that you don't have a single 
correct SHAsum that can be signed, communicated and compared easily.

You always need the full binary to compare to your rebuild.

The cleaner way is to use strip-nondeterminism to remove all these 
insignificant bits during build and make the resulting bit-reproducible 
output the official binary.


Ciao
Bernhard M.

[1] 
https://github.com/openSUSE/build-compare/commit/5cba04fb8def5d88423737a1a1957730e2217357


OpenPGP_signature
Description: OpenPGP digital signature


Re: Three bytes in a zip file

2023-04-07 Thread Bernhard M. Wiedemann via rb-general



On 06/04/2023 10.28, Larry Doolittle wrote:

I'm trying to make a process to generate byte-for-byte reproducible zip files.


Try adding the -X option to the zip call.
It will suppress adding of extended attributes (atime/ctime).
And with
https://github.com/distropatches/zip/commit/501ae4e93fd6fa2f7d20d00d1b011f9006802eae
it will also normalize mtime.


Ciao
Bernhard M.


OpenPGP_signature
Description: OpenPGP digital signature


Re: verifiable source-only bootstrap from scratch

2023-03-13 Thread Bernhard M. Wiedemann via rb-general



On 09/03/2023 23.34, Vagrant Cascadian wrote:

On 2023-03-08, aho...@0w.se wrote:

We seem to be the first project offering bootstrappable and verifiable
builds without any binary seeds.

The project's website is at [1]

...

[1] the site is available through the Tor/onion network
(for the advantages of convenient and privacy-friendly hosting) at
http://rbzfp7h25zcnmxu4wnxhespe64addpopah5ckfpdfyy4qetpziitp5qd.onion/


Is there a URL other than via tor .onion network to read up on what this
project is actually doing?

While I applaud and support the use of tor, exclusively using tor is a
bit of a surprise and seems to severely limit the scope of people who
will even read about it at all.

live well,
   vagrant


I created a mirror for easier access:

https://www.zq1.de/~bernhard/mirror/rbzfp7h25zcnmxu4wnxhespe64addpopah5ckfpdfyy4qetpziitp5qd.onion/

Ciao
Bernhard M.


OpenPGP_signature
Description: OpenPGP digital signature


Re: SBOMs - Anywhere?

2023-03-03 Thread Bernhard M. Wiedemann via rb-general



On 25/02/2023 16.56, Anthony Harrison wrote:
More tools are in the pipeline, including one to generate an SBOM from 
an installed platform distribution or package (currently works for 
Debian systems, work in progress for RPM based systems) and an audit 
tool. I hope to publish these in the next couple of weeks.


I want to mention that we can already generate [1] and publish [2] SBOMs 
in our Open-Build-Service to meet SLSA level4 requirements.



[1] https://github.com/openSUSE/obs-build/search?q=SBOM
[2] 
https://github.com/openSUSE/open-build-service/blob/1e051bb20fb385695399c79dd8c9920d5fa18273/src/backend/bs_regpush#L717


OpenPGP_signature
Description: OpenPGP digital signature


Re: How to talk to skeptics?

2022-12-21 Thread Bernhard M. Wiedemann via rb-general



On 18/12/2022 02.09, Martin via rb-general wrote:

Controlling hardware is essential


https://www.bunniestudios.com/blog/?p=5706

Covers the topic of why open-source hardware is not enough to build 
trustable devices.


TLDR: there are ways to subvert silicon that cannot be detected, even 
with a electron-microscope, even if you know where to look.


One way out are FPGAs wherein you place processor cores randomly, so 
attackers cannot know what to subvert at the time of fabrication.


However, this is orthogonal to reproducible+bootstrappable builds.

Ideally you have all of them, but having some of them, is better than 
having none.


Ciao
Bernhard M.


OpenPGP_signature
Description: OpenPGP digital signature


How to talk to skeptics?

2022-12-14 Thread Bernhard M. Wiedemann via rb-general

Hi,

a colleague of mine is rather skeptic towards bootstrapping and 
reproducible-builds.


E.g. he wrote

https://fy.blackhats.net.au/blog/html/2021/05/12/compiler_bootstrapping_can_we_trust_rust.html

and the effect can also be seen in his packaging such as
https://build.opensuse.org/package/show/openSUSE:Factory/rust1.65
that ships with two gigabytes of bootstrap compiler binaries for various 
architectures instead of using our existing rust packages of version N-1 
"because compilation takes twice as long".


He also once pointed me to
https://blog.cmpxchg8b.com/2020/07/you-dont-need-reproducible-builds.html

In the end, it would be useful to collect some well-worded / 
well-thought counter-arguments on r-b.o (if we don't have that already)


https://reproducible-builds.org/docs/buy-in/ could provide some input.

Any thoughts and/or volunteers?

Ciao
Bernhard M.


OpenPGP_signature
Description: OpenPGP digital signature


scons discussion

2022-09-26 Thread Bernhard M. Wiedemann via rb-general

Hi,

there is an interesting rb-related argument in a PR, but I want to avoid 
that too many rb-people hop in there and instead would appreciate, if 
you could contribute to a cohesive argument at

https://etherpad.opensuse.org/p/scons-rb-argument

that then gets posted there by 1 representative.


Ciao
Bernhard M.


OpenPGP_signature
Description: OpenPGP digital signature


Re: Fw: Build Reproducibility in Debian - Opinion Needed

2022-08-25 Thread Bernhard M. Wiedemann via rb-general
Muhammad Hassan wrote:
> Do you feel there is potential for detecting build unreproducibility 
> statically (without executing adversarial rebuilds)?

Yes, there are a number of potentially troublesome strings listed in
https://github.com/bmwiedemann/reproducibleopensuse/blob/master/howtodebug#L31

If one of these gets added, it may be harmless, but would warrant a
rebuild test or closer inspection of the source.


On 24/08/2022 19.37, Chris Lamb wrote:
> Other avenues requiring a single build would include all the instrumention
> approach (eg. strace/systemtap, etc.) taken by a few projects. I think
> Bernhard might be able to speak better on this, and there are some
> academic projects in this area as well.

My strace approach uses
https://github.com/bmwiedemann/reproducibleopensuse/blob/master/stracebuild
to trigger
https://github.com/bmwiedemann/reproducible-faketools/blob/master/bin/rpmbuild-strace

I use that to find where unreproducible files come from with
https://github.com/bmwiedemann/reproducibleopensuse/blob/master/autoprovenance

It seems, strace cannot see time syscalls - maybe because those do not
reach the kernel via the linux-vdso.so.1 shortcut.

It would be possible to see accesses to /dev/[u]random and readdir syscalls.


I have also played a bit with ptrace-based
https://github.com/dettrace/dettrace
but it needed regular updates as Linux keeps introducing new syscalls.



Ciao
Bernhard M.


OpenPGP_signature
Description: OpenPGP digital signature


Re: Fwd: enabling link time optimizations in package builds

2022-06-28 Thread Bernhard M. Wiedemann via rb-general


On 17/06/2022 11.12, Chris Lamb wrote:
> Hi Roland,
> 
>> would enabling LTO cause reproducible issues?
>> If I remember correctly, Bernhard mentioned some issues, which got 
>> 'solved' by using less parallel builds (-j1 or -j4?).
> 
> Good question. There was definitely at least one LTO-related issue in the
> past. Take, for instance, this bug report from 2015 about "FAT" LTO objects:
> 
>   https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66305
> 
> I don't know whether this is still valid and/or we would encounter it
> with Doko's proposal, however.

I think it can still be an issue.

We strip the unreproducible sections at the end of builds:
https://github.com/openSUSE/brp-check-suse/pull/29/files

The downside is that you don't get extra optimization when linking
external .a files, but such static linking is discouraged anyway for
maintainability.


The other thing we had to do was that we started to use -flto=auto
instead of -flto=$cpus
to avoid embedding this detail into debuginfo and such.

Ciao
Bernhard M.


OpenPGP_signature
Description: OpenPGP digital signature


Re: rb meetup at openSUSE conference in Nuremberg

2022-05-20 Thread Bernhard M. Wiedemann via rb-general


On 10/03/2022 04.23, Bernhard M. Wiedemann wrote:
> Hi,
> 
> I submitted a rb workshop session proposal for
> https://events.opensuse.org/conferences/oSC22
> 
> Even if that is not accepted, this conf would be an opportunity for a
> small meetup of rb people.
> 
> Who would be interested to join? Add yourself to
> https://dudle.inf.tu-dresden.de/NUE-rb-meetup-2022/
> (times are in CEST and are just rough indicators)

2 more weeks to go until openSUSE conference

Meanwhile the rb workshop is scheduled for 2022-06-03 10:00 CEST
https://events.opensuse.org/conferences/oSC22/program/proposals/3707

But the proposed meetup should be independent of that.
Feel free to still add yourself to the dudle linked above.

Ciao
Bernhard M.


OpenPGP_signature
Description: OpenPGP digital signature


Re: JDK 19+21 early-access build is reproducible

2022-05-08 Thread Bernhard M. Wiedemann via rb-general


On 06/05/2022 22.48, John Neffenger wrote:
> Starting yesterday, for the first time, the JDK can create reproducible
> builds of the JDK!

That is great news. Thank you John, Magnus and Andrew for taking care of
this.

I tried to get a double-build test working on openSUSE with
https://github.com/openjdk/jdk/releases/tag/jdk-19%2B21 to either
confirm the reproducibility or find some remaining diff for you,
however building JDK is still hard, so

https://build.opensuse.org/package/show/home:bmwiedemann:java/java-19-openjdk

errors out atm with

 checking for java...
/home/abuild/rpmbuild/BUILD/jdk-jdk-19-21/build/bootcycle-build/images/jdk/bin/java
 checking for javac...
/home/abuild/rpmbuild/BUILD/jdk-jdk-19-21/build/bootcycle-build/images/jdk/bin/javac
 checking for javah... no
 configure: error: Java 1.6 or later is required to build java-access-bridge


I found a javah in 1_8_0 but that is rejected as bootstrap java version
(needs to be 18 or 19)

Can you point me to the scripts that build your official Linux binaries
or do you have hints on how to fix my build?

You can also try this locally on Debian or openSUSE with an account from
https://idp-portal.suse.com/ and
osc co home:bmwiedemann:java/java-19-openjdk && cd $_
osc build --noservice standard


Ciao
Bernhard M.


OpenPGP_signature
Description: OpenPGP digital signature