This is an automated email from the ASF dual-hosted git repository.
slawrence pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/daffodil-infrastructure.git
The following commit(s) were added to refs/heads/main by this push:
new 0ccfc7e Modify check-release script to work with SBT and VS Code
releases
0ccfc7e is described below
commit 0ccfc7ef770463b9d73e75c4361c0a4bb84a695f
Author: Steve Lawrence <[email protected]>
AuthorDate: Fri Aug 29 13:23:04 2025 -0400
Modify check-release script to work with SBT and VS Code releases
The dist directory for SBT and VS Code releases includes an extra
'daffodil-sbt/' or 'daffodil-vscode/' directory, compared to normal
daffodil releases. And the value provided to --cut-dir does not consider
these extra directories, so downloaded directory tree structure doesn't
actaully match the structure created by the build-release container,
causing the reproducible diff to fail.
To avoid needing to detect and specify different cut dir values based on
the release type, this moves the wget logic to a new 'download_dir'
function that calculates the value of cut-dirs so that it only keeps the
last directory.
So for example, all dist artifacts are downloaded to a
directory matching the version (e.g. 1.0.0-rc1/) and all maven artifacts
are downloaded to a directory matching the maven profile (e.g.
orgapachedaffodil-1234/). Note that the build-container does not include
the maven profile directory, so after it is downloaded we move the
contents out of that directory.
This matches the structure that the build-release container creates
allowing reproducible diffs to work.
DAFFODIL-2971
---
containers/check-release/src/check-release.sh | 37 ++++++++++++++++++++++++---
1 file changed, 33 insertions(+), 4 deletions(-)
diff --git a/containers/check-release/src/check-release.sh
b/containers/check-release/src/check-release.sh
index 4118c52..fd67462 100755
--- a/containers/check-release/src/check-release.sh
+++ b/containers/check-release/src/check-release.sh
@@ -22,6 +22,31 @@ then
exit 1
fi
+download_dir() {
+ # force a trailing slash by removing a slash it if exists and then
adding
+ # it back. This is required for maven URLs to download the right
thing--it
+ # doesn't matter either way for dist urls
+ URL="${1%/}/"
+ # non-greedily delete the schema and domain part of the URL, giving just
+ # the path part
+ URL_PATH="${URL#*://*/}"
+ # we want to use --cut-dirs to ignore all directories but the final
target
+ # directory. This number is different depending on the URL, so we
extract
+ # and count the nubmer of slash-separated fields of the url path and
+ # subtract 2 (one because we want to keep the last field, and one
because
+ # there is an extra field because the path ends in a slash)
+ CUT_DIRS=$(echo "$URL_PATH" | awk -F/ '{print NF - 2}')
+ wget \
+ --recursive \
+ --level=inf \
+ -e robots=off \
+ --no-parent \
+ --no-host-directories \
+ --reject=index.html,robots.txt \
+ --cut-dirs=$CUT_DIRS \
+ "$URL"
+}
+
# URL of release candidate directory in dev/dist/, e.g.
https://dist.apache.org/repos/dist/dev/daffodil/1.0.0-rc1
DIST_URL=$1
@@ -45,7 +70,6 @@ require_command sha1sum
require_command sha512sum
require_command wget
-WGET="wget --recursive --level=inf -e robots=off --no-parent
--no-host-directories --reject=index.html,robots.txt"
RELEASE_DIR=release-download
DIST_DIR=$RELEASE_DIR/asf-dist
@@ -56,16 +80,21 @@ printf "\n==== Downloading Release Files ====\n"
# download dist/dev/ files
mkdir -p $DIST_DIR
pushd $DIST_DIR &>/dev/null
-$WGET --cut-dirs=4 $DIST_URL/
+download_dir $DIST_URL
popd &>/dev/null
-# download maven repository, delete nexus generated files
+# download maven repository, delete nexus generated files, and remove the
+# orgapachedaffodil-1234 dir since the build-release container does not have
+# this directory
if [ -n "$MAVEN_URL" ]
then
mkdir -p $MAVEN_DIR
pushd $MAVEN_DIR &>/dev/null
- $WGET --cut-dirs=3 $MAVEN_URL/
+ download_dir $MAVEN_URL
find . -type f \( -name 'archetype-catalog.xml' -o -name
'maven-metadata.xml*' \) -delete
+ REPO_DIR=(*/)
+ mv $REPO_DIR/* .
+ rmdir $REPO_DIR
popd &>/dev/null
fi
