This is an automated email from the ASF dual-hosted git repository.
snazy pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/polaris.git
The following commit(s) were added to refs/heads/main by this push:
new d8602f609 CI/Caching: Fix Gradle cache retention (#2604)
d8602f609 is described below
commit d8602f609b8137456ad11468a6533f5c531d74e0
Author: Robert Stupp <[email protected]>
AuthorDate: Thu Sep 18 13:41:40 2025 +0200
CI/Caching: Fix Gradle cache retention (#2604)
Older versions of Gradle's setup-gradle action did not actively trigger
stale cache entry cleanup, which is why there was a separate step "Trigger
Gradle home cleanup" in `gradle.yml`.
Nowadays, that action triggers a "noop build" to explicitly trigger stale
cache entry cleanup, but uses somewhat different defaults than [described
here](https://docs.gradle.org/current/userguide/directory_layout.html#dir:gradle_user_home:configure_cache_cleanup).
This change adds an explicit configuration for Gradle cache
cleanup/retention with reasonable values considering the total 10GB limit for
all GitHub caches per repository.
The change described above lead to a behavioral change, which evicts all
non-accessed cache entries within the current GH workflow job, which
effectively evicted all cache entries from dependent jobs - in other words: the
(build) cache was nearly empty, leading to full rebuilds.
This behavior could be observed in the output of the "Post Collect partial
Gradle build caches"-step in the log message "Build cache
(/home/runner/.gradle/caches/build-cache-1) removing files not accessed on or
after ..." showing the timestamp when the setup-gradle action was started.
---
.../actions/ci-incr-build-cache-prepare/action.yml | 25 ++++++++++++++++++++--
.../actions/ci-incr-build-cache-save/action.yml | 1 -
.github/workflows/gradle.yml | 7 ++----
3 files changed, 25 insertions(+), 8 deletions(-)
diff --git a/.github/actions/ci-incr-build-cache-prepare/action.yml
b/.github/actions/ci-incr-build-cache-prepare/action.yml
index f09d4992e..678dce27c 100644
--- a/.github/actions/ci-incr-build-cache-prepare/action.yml
+++ b/.github/actions/ci-incr-build-cache-prepare/action.yml
@@ -32,6 +32,27 @@ inputs:
runs:
using: "composite"
steps:
+ - name: Configure Gradle cache retention settings
+ # Without these settings, the Gradle Setup action will evict effectively
all "restored incremental" entries,
+ # as it will remove all cache entries created before the start-time of
the "Store Gradle Cache" job.
+ # The total size of GitHub caches for each repository is limited to
10GB, which means that we have to use
+ # more aggressive retention settings to keep the size of the Gradle
cache stored in GitHub at an appropriate
+ # size.
+ shell: bash
+ run: |
+ mkdir -p ~/.gradle/init.d
+ cat > ~/.gradle/init.d/cache-settings.gradle.kts <<!
+ beforeSettings {
+ caches {
+ releasedWrappers.setRemoveUnusedEntriesAfterDays(5)
+ snapshotWrappers.setRemoveUnusedEntriesAfterDays(1)
+ downloadedResources.setRemoveUnusedEntriesAfterDays(2)
+ createdResources.setRemoveUnusedEntriesAfterDays(5)
+ buildCache.setRemoveUnusedEntriesAfterDays(5)
+ }
+ }
+ !
+
- name: Prep env
shell: bash
run: |
@@ -71,12 +92,12 @@ runs:
if [[ -d ~/downloaded-artifacts/ ]] ; then
find ~/downloaded-artifacts/ -type f -name "ci-gradle-caches-*-${{
inputs.java-version }}.tar" | while read arch ; do
echo "Adding archive content from $arch ..."
- (cd ~/.gradle/caches/ ; tar xf $arch)
+ (cd ~/.gradle/caches/ ; tar xf $arch --atime-preserve)
done
else
echo "No previous build cache artifacts downloaded."
fi
-
+
date +%s > ~/caches-prepared-at-epoch
echo "::endgroup::"
diff --git a/.github/actions/ci-incr-build-cache-save/action.yml
b/.github/actions/ci-incr-build-cache-save/action.yml
index 4d6085c97..728ff6c01 100644
--- a/.github/actions/ci-incr-build-cache-save/action.yml
+++ b/.github/actions/ci-incr-build-cache-save/action.yml
@@ -47,7 +47,6 @@ runs:
-path './[0-9]*/kotlin-dsl/*' -or \
-path './jars-*/*' -or \
-path './modules-*/files-*/*' -or \
- -path './modules-*/files-*/*' -or \
-path './build-cache-*/*' \
')' | grep -v '[.]lock$' > ~/ci-gradle-caches-diff || true
echo "Identified $(wc -l < ~/ci-gradle-caches-diff) changed/added
files in caches/"
diff --git a/.github/workflows/gradle.yml b/.github/workflows/gradle.yml
index ddd44140a..153d3ce0f 100644
--- a/.github/workflows/gradle.yml
+++ b/.github/workflows/gradle.yml
@@ -160,9 +160,6 @@ jobs:
uses: ./.github/actions/ci-incr-build-cache-prepare
with:
cache-read-only: false
- - name: Trigger Gradle home cleanup
- env:
- DEVELOCITY_ACCESS_KEY: ${{ secrets.DEVELOCITY_ACCESS_KEY }}
- run: ./gradlew --no-daemon :showVersion
- # Note: the "Post Gradle invocation" archives the updated build cache.
+ # Note: the "Post Collect partial Gradle build caches"-step cleans up
stale cache entries
+ # and archives the updated build cache.
