This is an automated email from the ASF dual-hosted git repository.
zeroshade pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/arrow-go.git
The following commit(s) were added to refs/heads/main by this push:
new a01fe415 ci: retry and cache dependency downloads (#1279)
a01fe415 is described below
commit a01fe415098fb04c22796bb5ded6c2a5b395aaf2
Author: Matt Topol <[email protected]>
AuthorDate: Thu Sep 3 11:08:27 2026 -0400
ci: retry and cache dependency downloads (#1279)
### Rationale for this change
CI has intermittently failed while downloading Go modules and Apache
RAT, including HTTP/2 proxy failures and Maven Central HTTP 429
responses. These downloads currently fail on the first transient network
error, and RAT is downloaded again whenever the pre-commit environment
cache misses.
### What changes are included in this PR?
- Retry the Go module download in the Debian test image with bounded
backoff
- Retry RAT downloads on transient HTTP and transport failures
- Remove partial or checksum-invalid RAT artifacts
- Cache the checksum-verified RAT jar independently of pre-commit
environments
### Are these changes tested?
- `bash -n dev/release/run_rat.sh`
- `shellcheck dev/release/run_rat.sh`
- corrupt-RAT behavioral probe verifies checksum failure removes the
artifact
- `actionlint .github/workflows/lint.yml`
- `git diff --check`
### Are there any user-facing changes?
No. This only makes CI dependency acquisition more resilient.
Signed-off-by: Matt Topol <[email protected]>
---
.github/workflows/lint.yml | 5 +++++
ci/docker/debian-12.dockerfile | 7 ++++++-
dev/release/run_rat.sh | 6 ++++++
3 files changed, 17 insertions(+), 1 deletion(-)
diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml
index f52b8c31..3ad3ba15 100644
--- a/.github/workflows/lint.yml
+++ b/.github/workflows/lint.yml
@@ -57,6 +57,11 @@ jobs:
with:
path: ~/.cache/pre-commit
key: pre-commit-${{ hashFiles('.pre-commit-config.yaml') }}
+ - name: Cache Apache RAT
+ uses: actions/cache@v6
+ with:
+ path: dev/release/apache-rat-0.16.1.jar
+ key: apache-rat-0.16.1
- name: Run pre-commit
run: |
pre-commit run --all-files --color=always --show-diff-on-failure
diff --git a/ci/docker/debian-12.dockerfile b/ci/docker/debian-12.dockerfile
index e2625995..95187272 100644
--- a/ci/docker/debian-12.dockerfile
+++ b/ci/docker/debian-12.dockerfile
@@ -21,4 +21,9 @@ FROM ${arch}/golang:${go}-bookworm
# Copy the go.mod and go.sum over and pre-download all the dependencies
COPY . /arrow-go
-RUN cd /arrow-go && go mod download github.com/apache/arrow-go/v18@latest
+RUN cd /arrow-go && \
+ for attempt in 1 2 3 4 5; do \
+ go mod download github.com/apache/arrow-go/v18@latest && exit 0; \
+ if [ "${attempt}" -eq 5 ]; then exit 1; fi; \
+ sleep $((attempt * 2)); \
+ done
diff --git a/dev/release/run_rat.sh b/dev/release/run_rat.sh
index afecfc20..3f130fea 100755
--- a/dev/release/run_rat.sh
+++ b/dev/release/run_rat.sh
@@ -29,6 +29,11 @@ if [ ! -f "${RAT_JAR}" ]; then
curl \
--fail \
--output "${RAT_JAR}" \
+ --remove-on-error \
+ --retry 5 \
+ --retry-all-errors \
+ --retry-delay 2 \
+ --retry-max-time 60 \
--show-error \
--silent \
https://repo1.maven.org/maven2/org/apache/rat/apache-rat/${RAT_VERSION}/apache-rat-${RAT_VERSION}.jar
@@ -48,6 +53,7 @@ if actual != expected:
raise SystemExit(1)
PY
echo "checksum verification failed for ${RAT_JAR}" >&2
+ rm -f "${RAT_JAR}"
exit 1
fi
