Re: [Rpm-maint] [rpm-software-management/rpm] rpm-4.18.0 embeds build machine CPU count (Issue #2343)
Yes, there will be any number of ways the parsed spec inclusion can trip up bit-for-bit differences. "Breakage" is a strange and strong word for it, another angle to look at it is to use it as a tool to help reproducability: if the parsed spec (which is what the build will actually use, after all) differs it's acts as a warning that requires investigation. Some of which will be false positives, like mktemp. -- Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/2343#issuecomment-1689379487 You are receiving this because you are subscribed to this thread. Message ID: ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] rpm-4.18.0 embeds build machine CPU count (Issue #2343)
found another breakage:
`sed -i -e 's/-j2/%{?_smp_mflags}/' setup.py`
in
https://github.com/bmwiedemann/openSUSE/blob/master/packages/p/python-python-poppler/python-python-poppler.spec#L61
--
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/2343#issuecomment-1689242765
You are receiving this because you are subscribed to this thread.
Message ID: ___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] rpm-4.18.0 embeds build machine CPU count (Issue #2343)
Found another reproducibility problem with expanded .spec in
https://code.opensuse.org/package/texlive/blob/d820a867c61524278af352b4febbb115e9dad08f/f/texlive.spec#_287
```spec
%{expand: %%global options %(mktemp /tmp/texlive-opts.)}
```
--
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/2343#issuecomment-1678814874
You are receiving this because you are subscribed to this thread.
Message ID: ___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] rpm-4.18.0 embeds build machine CPU count (Issue #2343)
Closed #2343 as completed. -- Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/2343#event-8886283587 You are receiving this because you are subscribed to this thread. Message ID: ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] rpm-4.18.0 embeds build machine CPU count (Issue #2343)
The reported and most glaring issue with %{_smp_mflags} was addressed in
4.18.x, closing but acknowledging that we may need something further around
this.
--
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/2343#issuecomment-1490125155
You are receiving this because you are subscribed to this thread.
Message ID: ___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] rpm-4.18.0 embeds build machine CPU count (Issue #2343)
> binary rpm files embed the .src.rpm header checksum, so suffer as well. Ugh, I hadn't realized the src.rpm header md5 (another ugh) ends up in the binary headers too. It only happens with -ba (iirc) so not all build-systems exhibit that, but still. This would be nice case for placing the parsed spec outside the checksummed part of the header, but that in turn causes other problems... -- Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/2343#issuecomment-1481118723 You are receiving this because you are subscribed to this thread. Message ID: ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] rpm-4.18.0 embeds build machine CPU count (Issue #2343)
And more trouble from https://codeberg.org/cunix/vendored_licenses_packager/src/branch/main/macros.vendored_licenses_packager#L18 that injects a random tmp path into the .src.rpm of [dnscrypt-proxy](https://code.opensuse.org/package/dnscrypt-proxy/tree/master) -- Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/2343#issuecomment-1420449964 You are receiving this because you are subscribed to this thread. Message ID: ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] rpm-4.18.0 embeds build machine CPU count (Issue #2343)
Found another problematic case:
https://code.opensuse.org/package/ocaml-rpm-macros/blob/819e56/f/ocaml-rpm-macros.spec#_464
`'%%{?_smp_mflags}'`
becomes
`'-j${RPM_BUILD_NCPUS}'` and the variable does not get expanded now, causing a
compilation failure. I guess, using double-quotes should fix it.
--
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/2343#issuecomment-1407598501
You are receiving this because you are subscribed to this thread.
Message ID: ___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] rpm-4.18.0 embeds build machine CPU count (Issue #2343)
Yep, it's not exactly a big change: d97d7b71de158660eb96b4f11d40b6626b85521a -- Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/2343#issuecomment-1398406061 You are receiving this because you are subscribed to this thread. Message ID: ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] rpm-4.18.0 embeds build machine CPU count (Issue #2343)
Backport `$RPM_BUILD_NCPUS`. -- Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/2343#issuecomment-1398345293 You are receiving this because you are subscribed to this thread. Message ID: ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] rpm-4.18.0 embeds build machine CPU count (Issue #2343)
It seems, `$RPM_BUILD_NCPUS` is not set in our older distributions that have rpm-4.14.3. What would be the best way to have https://github.com/mesonbuild/meson/blob/c754f9076/data/macros.meson#L43 be reproducible on both 4.14 and 4.18 ? -- Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/2343#issuecomment-1398053168 You are receiving this because you are subscribed to this thread. Message ID: ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] rpm-4.18.0 embeds build machine CPU count (Issue #2343)
That would break the macro for anybody using it outside the build scriptlets, which is an entirely legit thing to do. You'll need to fix those specs instead. -- Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/2343#issuecomment-1381471213 You are receiving this because you are subscribed to this thread. Message ID: ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] rpm-4.18.0 embeds build machine CPU count (Issue #2343)
There are four .spec files in openSUSE, that use `%{_smp_build_ncpus}`
directly, e.g.
https://code.opensuse.org/package/python3-pyside6/blob/ad8e8792e6ed522cb99332637de53ce6ff5b93f1/f/python3-pyside6.spec#_226
Maybe instead of 0576d24756fe975d890f5535a21cfdfd35fc2ca4, we could redefine
`%_smp_build_ncpus` to `$RPM_BUILD_NCPUS` ?
--
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/2343#issuecomment-1381436669
You are receiving this because you are subscribed to this thread.
Message ID: ___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] rpm-4.18.0 embeds build machine CPU count (Issue #2343)
Yes. It came from https://github.com/openSUSE/obs-build/blob/master/build-recipe#L234 -- Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/2343#issuecomment-1379094502 You are receiving this because you are subscribed to this thread. Message ID: ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] rpm-4.18.0 embeds build machine CPU count (Issue #2343)
> Right something writes it into ~/.rpmmacros . That is our problem then and
> the `-j%{_RPM_BUILD_NCPUS} ` rpm patch should still be good.
This is probably `obs-build`, which redefines a bunch of stuff in
`~/.rpmmacros` when it shouldn't.
--
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/2343#issuecomment-1379088468
You are receiving this because you are subscribed to this thread.
Message ID: ___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] rpm-4.18.0 embeds build machine CPU count (Issue #2343)
To whomever ends up doing 4.18.1 release: this needs commits 5049fc701561b258b722b3400460d8828ce9e64e and 0576d24756fe975d890f5535a21cfdfd35fc2ca4 -- Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/2343#issuecomment-1378607157 You are receiving this because you are subscribed to this thread. Message ID: ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] rpm-4.18.0 embeds build machine CPU count (Issue #2343)
Oh. That's installplatform being over-eager (`-e '/\${\w*:-/!s,\${,%{_,' `
presumably). So better to include the move to macros then, it's the right thing
anyhow.
--
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/2343#issuecomment-1378563459
You are receiving this because you are subscribed to this thread.
Message ID: ___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] rpm-4.18.0 embeds build machine CPU count (Issue #2343)
I have it with `-j${RPM_BUILD_NCPUS}` in
https://build.opensuse.org/package/view_file/home:bmwiedemann:reproducible/rpm/0576d24756f.patch
, but somehow it gets replaced by `%_smp_mflags -j%{_RPM_BUILD_NCPUS}` in
platform files, which then breaks because the macro is unknown.
With the move to `macros.in`, it works. Is there some special rewriting?
--
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/2343#issuecomment-1378546360
You are receiving this because you are subscribed to this thread.
Message ID: ___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] rpm-4.18.0 embeds build machine CPU count (Issue #2343)
> `-j%{_RPM_BUILD_NCPUS}` rpm patch should still be good.
Note, it's `-j${_RPM_BUILD_NCPUS}`, not with `%`
--
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/2343#issuecomment-1378518291
You are receiving this because you are subscribed to this thread.
Message ID: ___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] rpm-4.18.0 embeds build machine CPU count (Issue #2343)
Ack. rpmdev-setuptree < 8.6 liked to put a value in there, but that was dropped in 2015 so it's a fairly old thing, something else seems more likely. -- Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/2343#issuecomment-1378516913 You are receiving this because you are subscribed to this thread. Message ID: ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] rpm-4.18.0 embeds build machine CPU count (Issue #2343)
Right something writes it into ~/.rpmmacros . That is our problem then and the
`-j%{_RPM_BUILD_NCPUS}
` rpm patch should still be good.
--
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/2343#issuecomment-1378508483
You are receiving this because you are subscribed to this thread.
Message ID: ___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] rpm-4.18.0 embeds build machine CPU count (Issue #2343)
As mentioned in the PR, `-j${_RPM_BUILD_NCPUS}` cannot possibly expand to
-j because the macro engine does not expand environment variables from
by `$`. You appear to have a redefinition of %_smp_mflags someplace,
~/.rpmmacros being one of the possibilities.
--
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/2343#issuecomment-1378497385
You are receiving this because you are subscribed to this thread.
Message ID: ___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] rpm-4.18.0 embeds build machine CPU count (Issue #2343)
I checked git log and found some related commits. #2344 now does the trick for me. -- Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/2343#issuecomment-1378203226 You are receiving this because you are subscribed to this thread. Message ID: ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] rpm-4.18.0 embeds build machine CPU count (Issue #2343)
I tried
```diff
===
--- rpm-4.18.0.orig/platform.in
+++ rpm-4.18.0/platform.in
@@ -57,7 +57,7 @@
if [ -n "$ncpus_max" ] && [ "$ncpus_max" -gt 0 ] && [
"$RPM_BUILD_NCPUS" -gt "$ncpus_max" ]; then RPM_BUILD_NCPUS="$ncpus_max"; fi;
\\\
echo "$RPM_BUILD_NCPUS";)
-%_smp_mflags -j%{_smp_build_ncpus}
+%_smp_mflags -j${RPM_BUILD_NCPUS}
# Maximum number of threads to use when building, 0 for unlimited
#%_smp_nthreads_max 0
```
But somehow I still get this diff
```diff
-make -j4
+make -j1
```
the produced `/usr/lib/rpm/platform/x86_64-linux/macros` has
```
%_smp_mflags -j%{_RPM_BUILD_NCPUS}
```
Why is that?
--
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/2343#issuecomment-1377915374
You are receiving this because you are subscribed to this thread.
Message ID: ___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [rpm-software-management/rpm] rpm-4.18.0 embeds build machine CPU count (Issue #2343)
Right. This is very similar to the LTO case
(7faf8eda1358f8a877b9b3d6e1197b814e80b50b), with probably a similar solution.
The simple and sane solution is to have _smp_mflags expand to
`-j${RPM_BUILD_NCPUS}`
Which ... we already did for slightly different reasons, but apparently I
completely forgot about it in the meanwhile :laughing: - see
0576d24756fe975d890f5535a21cfdfd35fc2ca4. We just need to backport it to 4.18.
--
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/2343#issuecomment-1376930039
You are receiving this because you are subscribed to this thread.
Message ID: ___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint
[Rpm-maint] [rpm-software-management/rpm] rpm-4.18.0 embeds build machine CPU count (Issue #2343)
While working on [reproducible builds](https://reproducible-builds.org/) for
[openSUSE](https://en.opensuse.org/openSUSE:Reproducible_Builds), I found that
the recent upgrade to rpm-4.18.0
with #2047 / #1241 caused spec files with
```bash
make %{?_smp_mflags}
```
to embed the build machine CPU count into .src.rpm files, making them hard to
bit-reproduce.
binary rpm files embed the .src.rpm header checksum, so suffer as well.
It might also introduce other hard-to-reproduce details, but I have not looked
for these, yet.
--
Reply to this email directly or view it on GitHub:
https://github.com/rpm-software-management/rpm/issues/2343
You are receiving this because you are subscribed to this thread.
Message ID: ___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint
