This is an automated email from the ASF dual-hosted git repository.
slawrence pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/daffodil-infrastructure.git
The following commit(s) were added to refs/heads/main by this push:
new b58aefa Fix check-release RPM artifact verification
b58aefa is described below
commit b58aefa0a0cd68a47e89116a04576be7da8f4b64
Author: Steve Lawrence <[email protected]>
AuthorDate: Wed Apr 9 14:09:50 2025 -0400
Fix check-release RPM artifact verification
- RPM maintains a separate keychain for public gpg keys. When building
the check-release container, run rpm --import to import keys to that
keychain
- Modify a check-release message to make it more clear it is verifying
embedded RPM signatures and not the detached .asc signatures
- The rpm -K option succeeds even if the RPM does not have a gpg
signature. To require that an RPM both has an embedded gpg signature and
that it is valid, we grep for a specific string output only when both
conditions hold
- When checking reproducibility, we delete the signature embedded in
RPMs. But this means if you run the script again that signure will be
missing and signature/sha512 verification will fail. To prevent this,
we backup RPMs to a temporary directory prior to deleting signatures
and then restore them when the reproducible build check is done
DAFFODIL-2971
---
containers/check-release/Dockerfile | 3 ++-
containers/check-release/src/check-release.sh | 30 +++++++++++++++++++++------
2 files changed, 26 insertions(+), 7 deletions(-)
diff --git a/containers/check-release/Dockerfile
b/containers/check-release/Dockerfile
index 14b2778..a1c8d40 100644
--- a/containers/check-release/Dockerfile
+++ b/containers/check-release/Dockerfile
@@ -24,7 +24,8 @@ RUN \
RUN \
wget https://downloads.apache.org/daffodil/KEYS && \
- gpg --import KEYS
+ gpg --import KEYS && \
+ rpm --import KEYS
# Install and set the entrypoint
COPY src/check-release.sh /usr/bin/daffodil-check-release
diff --git a/containers/check-release/src/check-release.sh
b/containers/check-release/src/check-release.sh
index 7f96816..225f266 100755
--- a/containers/check-release/src/check-release.sh
+++ b/containers/check-release/src/check-release.sh
@@ -96,9 +96,20 @@ printf "\n==== Dist GPG Signatures ====\n"
find $DIST_DIR -type f ! -name '*.sha512' ! -name '*.asc' \
-exec bash -c "gpg --verify '{}.asc' '{}' $PRINT_FIND_RESULT" \;
-printf "\n==== RPM Signatures ====\n"
+printf "\n==== RPM Embedded Signatures ====\n"
+# The "rpm -K ..." command is used to verify that embedded digests and/or
+# signatures of an RPM are correct, but it does not require that either
+# actually exists. The format of its output is
+#
+# <rpm_name>: (digests)? (signatures)? [OK|NOT OK]
+#
+# where "digests" and "signatures" are optional (depending on if the RPM has
+# embedded digests/signatures) and "OK" is output if all embedded digests and
+# signatures are valid, or "NOT OK" otherwise. We require that released RPMs
+# have both embedded signatures and digests and that they are all valid, so we
+# ensure the output of rpm -K contains the expect string that indicates this.
find $DIST_DIR -type f -name '*.rpm' \
- -exec bash -c "rpm -K '{}' $PRINT_FIND_RESULT" \;
+ -exec bash -c "rpm -K '{}' | grep 'digests signatures OK'
$PRINT_FIND_RESULT" \;
if [ -n "$MAVEN_URL" ]
then
@@ -122,10 +133,13 @@ then
exit 0
fi
-printf "\n==== Calculating Differences ====\n"
-
-# The released rpm file has an embedded signature, deleting the signature
-# should cause the RPMs to be byte-for-byte the same
+# RPM files have an embedded signature which makes reproducibility checking
+# difficult since locally built RPMs will not have the embedded signature.
+# However, the RPMs should be identical if we delete that signature. So we
+# create a backup of the original RPM files, delete the embedded signature,
+# run the diff command, and then restore the backups.
+BACKUP_DIR=$(mktemp -d)
+find $DIST_DIR -name '*.rpm' -exec cp --parents {} $BACKUP_DIR \;
find $DIST_DIR -name '*.rpm' -execdir rpmsign --delsign {} \; &>/dev/null
# Reasons for excluding files from the diff check:
@@ -143,3 +157,7 @@ DIFF=$(diff \
--exclude=*.asc.sha1 \
$RELEASE_DIR/ $LOCAL_RELEASE_DIR/)
[ $? -eq 0 ] && echo -e "$PASS no differences found" || (echo "$DIFF" | xargs
-I {} echo -e "$FAIL {}")
+
+# restore and delete the backup directory
+cp -R $BACKUP_DIR/. .
+rm -rf $BACKUP_DIR
