Re: [PATCH cygport] Add repro-check command
On 01/03/2024 19:16, Christian Franke via Cygwin-apps wrote: Christian Franke wrote: This could be used to check whether a package is possibly reproducible. Then it could make sense to add a reasonable SOURCE_DATE_EPOCH value to the cygport file. [...] An enhanced version of the patch is attached. The build and diff could now be run also individually and the diff report includes individual files from the packages. As a side effect, this enables another use case: Check whether changes to cygport only change the expected files. $ cygport project.cygport all repro-check ... *** Info: Rebuild produced identical packages Applied. Thanks!
Re: [PATCH cygport] Add repro-check command
Christian Franke wrote:
This could be used to check whether a package is possibly
reproducible. Then it could make sense to add a reasonable
SOURCE_DATE_EPOCH value to the cygport file.
Example:
$ export SOURCE_DATE_EPOCH=$(date +%s)
$ cygport project.cygport all repro-check
...
*** Info: Build reproducibility test succeeded
$ TZ=UTC cygport project.cygport repro-check
...
*** Info: Build reproducibility test succeeded
$ unset SOURCE_DATE_EPOCH
$ cygport project.cygport repro-check
...
*** ERROR: Build reproducibility test failed
An enhanced version of the patch is attached. The build and diff could
now be run also individually and the diff report includes individual
files from the packages.
As a side effect, this enables another use case: Check whether changes
to cygport only change the expected files.
$ cygport project.cygport all repro-check
...
*** Info: Rebuild produced identical packages
$ editor project.cygport
... Change some comments ...
$ cygport project.cygport all repro-diff
...
Differing files found:
! dist/project/project-1.2-3-src.tar.xz
! spkg/project-1.2-3.src/project.cygport
*** ERROR: Rebuild differs from original
--
Regards,
Christian
From 152a21dfad4c786cff1712e6aa1c33f2db0b6a75 Mon Sep 17 00:00:00 2001
From: Christian Franke
Date: Fri, 1 Mar 2024 19:46:50 +0100
Subject: [PATCH] Add repro-build, repro-diff und repro-check commands
These commands check for reproducibility of distribution packages.
The repro-build command unpacks the source package from the dist
directory to the temp directory and performs a nested rebuild of
the packages there.
The repro-diff command compares original and rebuild packages.
If different, a report about individual differing files in dist,
inst and spkg directories is printed.
The repro-check command combines both commands.
---
README | 3 +++
bin/cygport.in | 17 +
lib/help.cygpart| 3 +++
lib/pkg_pkg.cygpart | 58 -
4 files changed, 80 insertions(+), 1 deletion(-)
diff --git a/README b/README
index fd16df6b..3c9e4d4a 100644
--- a/README
+++ b/README
@@ -163,6 +163,9 @@ Other COMMANDs are meant primarily for maintainers:
diff - write a patch file capturing changes to source in the working
directory
stage- as upload, but don't request processing of uploaded packages
announce - compose and send a package announcement
+repro-build - rebuild from created source package to temp directory
+repro-diff - check whether packages from original and rebuild differ
+repro-check - run repro-build and repro-diff
The standard arguments --help or --version may also be passed to cygport.
diff --git a/bin/cygport.in b/bin/cygport.in
index 5fc89eaf..a2c2b5a3 100755
--- a/bin/cygport.in
+++ b/bin/cygport.in
@@ -29,6 +29,10 @@ set -e;
#
+# Preserve original environment for repro-check command
+declare -r _cygport_orig_env=$(export)
+declare -r _cygport_orig_pwd=$(pwd)
+
# for regexes, sort, etc.
export LC_COLLATE=C
@@ -784,6 +788,19 @@ do
test ${PIPESTATUS[0]} -eq 0
_status=$?;
;;
+ repro-build)
+ __pkg_repro_build
+ _status=$?
+ ;;
+ repro-diff)
+ __pkg_repro_diff
+ _status=$?
+ ;;
+ repro-check)
+ __pkg_repro_build && \
+ __pkg_repro_diff
+ _status=$?
+ ;;
help)
__show_help;
exit 0;
diff --git a/lib/help.cygpart b/lib/help.cygpart
index a7f30f7a..d28fd7bb 100644
--- a/lib/help.cygpart
+++ b/lib/help.cygpart
@@ -56,6 +56,9 @@ __show_help() {
finishdelete the working directory
all run prep, compile, install and package
all-test run prep, compile, install and package-test
+ repro-build rebuild from created source package to temp
directory
+ repro-diffcheck whether packages from original and
rebuild differ
+ repro-check run repro-build and repro-diff
See the included README file for further documentation.
diff --git a/lib/pkg_pkg.cygpart b/lib/pkg_pkg.cygpart
index 756a687c..3c531c0e 100644
--- a/lib/pkg_pkg.cygpart
+++ b/lib/pkg_pkg.cygpart
@@ -992,6 +992,62 @@ _EOF
fi
}
+__pkg_repro_build() {
+ local srcpkg=${distdir}/${PN}/${PF}-src.tar.${TAR_COMPRESSION_EXT}
+ local t_spkgdir=${T}/${spkgdir##*/}
+ local t_workdir=${t_spkgdir}/${PF}.${ARCH}
+ local t_cygport="cygport ${cygportfile} finish all"
+ local rc
+
+ __stage "Rebuilding"
+ __step
[PATCH cygport] Add repro-check command
This could be used to check whether a package is possibly reproducible.
Then it could make sense to add a reasonable SOURCE_DATE_EPOCH value to
the cygport file.
Example:
$ export SOURCE_DATE_EPOCH=$(date +%s)
$ cygport project.cygport all repro-check
...
*** Info: Build reproducibility test succeeded
$ TZ=UTC cygport project.cygport repro-check
...
*** Info: Build reproducibility test succeeded
$ unset SOURCE_DATE_EPOCH
$ cygport project.cygport repro-check
...
*** ERROR: Build reproducibility test failed
--
Regards,
Christian
From 97f518478dac722647b8a423068f2a5461c82f19 Mon Sep 17 00:00:00 2001
From: Christian Franke
Date: Sun, 18 Feb 2024 16:33:07 +0100
Subject: [PATCH] Add repro-check command
This command checks for reproducibility of distribution packages.
The source package from the dist directory is unpacked to the
temp directory. A nested rebuild of the packages is run there.
If successful, original and rebuild packages are compared and the
result is reported.
---
README | 1 +
bin/cygport.in | 8
lib/help.cygpart| 1 +
lib/pkg_pkg.cygpart | 42 +-
4 files changed, 51 insertions(+), 1 deletion(-)
diff --git a/README b/README
index fd16df6b..fec46b13 100644
--- a/README
+++ b/README
@@ -163,6 +163,7 @@ Other COMMANDs are meant primarily for maintainers:
diff - write a patch file capturing changes to source in the working
directory
stage- as upload, but don't request processing of uploaded packages
announce - compose and send a package announcement
+repro-check - check whether a rebuild produces binary identical packages
The standard arguments --help or --version may also be passed to cygport.
diff --git a/bin/cygport.in b/bin/cygport.in
index 5fc89eaf..6acbc85b 100755
--- a/bin/cygport.in
+++ b/bin/cygport.in
@@ -29,6 +29,10 @@ set -e;
#
+# Preserve original environment for repro-check command
+declare -r _cygport_orig_env=$(export)
+declare -r _cygport_orig_pwd=$(pwd)
+
# for regexes, sort, etc.
export LC_COLLATE=C
@@ -784,6 +788,10 @@ do
test ${PIPESTATUS[0]} -eq 0
_status=$?;
;;
+ repro-check)
+ __pkg_repro_check
+ _status=$?
+ ;;
help)
__show_help;
exit 0;
diff --git a/lib/help.cygpart b/lib/help.cygpart
index a7f30f7a..d851762e 100644
--- a/lib/help.cygpart
+++ b/lib/help.cygpart
@@ -56,6 +56,7 @@ __show_help() {
finishdelete the working directory
all run prep, compile, install and package
all-test run prep, compile, install and package-test
+ repro-check check whether a rebuild produces binary
identical packages
See the included README file for further documentation.
diff --git a/lib/pkg_pkg.cygpart b/lib/pkg_pkg.cygpart
index 756a687c..719ffcd1 100644
--- a/lib/pkg_pkg.cygpart
+++ b/lib/pkg_pkg.cygpart
@@ -992,6 +992,46 @@ _EOF
fi
}
+__pkg_repro_check() {
+ local rc srcpkg t_cygport t_spkgdir
+
+ srcpkg=${distdir}/${PN}/${PF}-src.tar.${TAR_COMPRESSION_EXT}
+ t_spkgdir=${T}/${spkgdir##*/}
+
+ echo
+ __stage "Checking reproducibility of"
+
+ echo
+ __step "Unpacking ${srcpkg}"
+ [ -f "${srcpkg}" ] || error "Packages not built yet"
+ tar xf ${srcpkg} -C ${T} || error "tar xf ${srcpkg} -C ${T} failed"
+
+ echo
+ __step "Rebuilding in ${t_spkgdir}"
+ t_cygport="cygport ${cygportfile} finish all"
+ echo "${_cygport_orig_env}" > ${T}/.cygport_orig_env
+ __step "=== Start: ${t_cygport} ="
+
+ # Start nested cygport with original environment in temp directory
+ rc=0
+ env --chdir=${_cygport_orig_pwd} --ignore-environment /bin/bash -c \
+ "source ${T}/.cygport_orig_env && cd ${t_spkgdir} &&
${t_cygport}" \
+ || rc=$?
+
+ __step "=== Done: ${t_cygport} (exit $rc) ="
+ echo
+ [ $rc = 0 ] || error "Rebuild in ${t_spkgdir} failed"
+
+ __step "Comparing original and rebuilt packages"
+ if ! diff -qr ${distdir} ${t_spkgdir}/${PF}.${ARCH}/dist
+ then
+ echo
+ error "Build reproducibility test failed"
+ fi
+ echo
+ inform "Build reproducibility test succeeded"
+}
+
# protect functions
readonly -f __pkg_binpkg __pkg_diff __gpg_sign __pkg_srcpkg __pkg_dist \
-__squeeze_whitespace __tar
+__pkg_repro_check __squeeze_whitespace __tar
--
2.43.0
