This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch refresh-image-registry-cache-workflow in repository https://gitbox.apache.org/repos/asf/airflow.git
commit dc2c305c16a73449b2990f8050c04b9c14c438a0 Author: Jarek Potiuk <[email protected]> AuthorDate: Wed Aug 5 18:29:18 2026 +0800 Exempt a manually started cache refresh from the push cancellation A push-triggered refresh should give way to the next one -- it is building the cache of a commit that is no longer the tip. A manual run should not: it is started because the branch has no cache and the push runs are the thing that keeps getting cancelled. --- .github/workflows/refresh-image-registry-cache.yml | 25 ++++++++++++++-------- ...UALLY_GENERATING_IMAGE_CACHE_AND_CONSTRAINTS.md | 5 +++++ 2 files changed, 21 insertions(+), 9 deletions(-) diff --git a/.github/workflows/refresh-image-registry-cache.yml b/.github/workflows/refresh-image-registry-cache.yml index c78c3fc8069..23b3e6221dc 100644 --- a/.github/workflows/refresh-image-registry-cache.yml +++ b/.github/workflows/refresh-image-registry-cache.yml @@ -19,13 +19,13 @@ name: Refresh image registry cache # Refreshes the CI image cache in the GitHub registry, in a run of its own. # -# It has to be its own workflow rather than a job inside `ci-amd.yml` / `ci-arm.yml`, because -# those cancel in progress on the next push and a job cannot opt out of its run being cancelled. -# A cache refresh is the one job that must not be cancelled: with no cache the branch builds cold -# (~20 minutes per Python version), which widens the window in which the next push cancels the -# run, which leaves the cache missing. `v3-3-test` sat in that loop -- its `linux/amd64` cache -# covered the default Python version alone, the single matrix entry that finished before the next -# push killed the other four seconds later. +# It has to be its own workflow rather than a job inside `ci-amd.yml` / `ci-arm.yml`, because a +# job cannot opt out of its run being cancelled, and those runs are cancelled by the next push. +# With no cache a branch builds cold (~20 minutes per Python version), which widens the window in +# which the next push cancels the run, which leaves the cache missing. `v3-3-test` sat in that +# loop -- its `linux/amd64` cache covered the default Python version alone, the single matrix +# entry that finished before the next push killed the other four seconds later. Being a separate +# workflow is what lets a manual run escape that (see the concurrency group below). # # Every Python version is refreshed in both trigger paths. That is what makes the cache useful: # a version with no cache entry builds from scratch on every run, whichever way it got skipped. @@ -51,8 +51,15 @@ on: # yamllint disable-line rule:truthy permissions: contents: read concurrency: - group: refresh-image-registry-cache-${{ github.ref }} - cancel-in-progress: false + # Push runs are never cancelled -- being cancelled by the next merge is the whole failure this + # workflow exists to escape, so a newer one queues behind the one in flight rather than + # replacing it. GitHub keeps at most one run pending per group, so the queue cannot grow. + # + # Manual runs are grouped separately, which is what keeps a merge from cancelling one, and they + # do supersede each other: firing the workflow again for the same branch says the run in flight + # is not the one you want. + group: refresh-image-registry-cache-${{ github.ref }}-${{ github.event_name }} + cancel-in-progress: ${{ github.event_name == 'workflow_dispatch' }} jobs: build-info: name: "Build info" diff --git a/dev/MANUALLY_GENERATING_IMAGE_CACHE_AND_CONSTRAINTS.md b/dev/MANUALLY_GENERATING_IMAGE_CACHE_AND_CONSTRAINTS.md index 6ebbf9e964f..cd07deec80b 100644 --- a/dev/MANUALLY_GENERATING_IMAGE_CACHE_AND_CONSTRAINTS.md +++ b/dev/MANUALLY_GENERATING_IMAGE_CACHE_AND_CONSTRAINTS.md @@ -105,6 +105,11 @@ Prefer this over the local route below. The workflow needs no buildx / qemu setu does not depend on your upload bandwidth, it always covers every Python version, and it runs with the registry credentials the CI already has, so you do not need to be logged in to `ghcr.io` as a committer. +A run you start by hand is not cancelled by merges landing while it works -- it uses a concurrency +group of its own for that reason. Starting it again for the same branch does replace the run in +flight. Push-triggered refreshes are never cancelled either; a newer one waits for the one in +flight to finish. + The workflow definition has to exist on the branch you select, so a release branch needs the workflow backported before it can be refreshed there.
