This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch breeze-install-from-lock in repository https://gitbox.apache.org/repos/asf/airflow.git
commit b14995d939bcd32dbc34064b1e8e3f198a3d2cb8 Author: Jarek Potiuk <[email protected]> AuthorDate: Wed Aug 26 19:21:00 2026 +0200 Run breeze from the locked dev/breeze environment Breeze was installed by resolving dev/breeze/pyproject.toml against the package index on every fresh environment — `uvx --from ./dev/breeze` in the shim, `uv tool install --editable ./dev/breeze` in CI. Neither reads dev/breeze/uv.lock, and neither applies the `[tool.uv] exclude-newer` buffer declared beside it, which governs project operations only. The committed lock recorded nothing, so an upstream release could change what breeze runs with no commit in this repository. click 8.5.0 showed the cost. It added a `help` field to the argument info dict breeze hashes to detect command drift, so within the hour of its release every CI job resolved it, every command taking a positional argument hashed differently from its committed value, and static checks went red on every open PR regardless of what it touched. The lock said click 8.4.2 throughout. The command-output files move back to the versions the lock produces; the next lock upgrade regenerates them together with the dependency bump that causes them. --- .github/workflows/basic-tests.yml | 4 +- AGENTS.md | 2 +- contributing-docs/03_contributors_quick_start.rst | 7 +- .../contributors_quick_start_gitpod.rst | 5 +- dev/README_AIRFLOW3_DEV.md | 3 +- dev/README_RELEASE_AIRFLOW.md | 2 +- dev/breeze/README.md | 7 +- dev/breeze/doc/01_installation.rst | 7 +- .../doc/adr/0016-use-uv-tool-to-install-breeze.md | 2 +- ...017-use-uvx-to-run-breeze-from-local-sources.md | 74 +++++++++++++++------- dev/breeze/doc/images/output_build-docs.txt | 2 +- dev/breeze/doc/images/output_ci-image_pull.txt | 2 +- dev/breeze/doc/images/output_ci-image_verify.txt | 2 +- dev/breeze/doc/images/output_exec.txt | 2 +- .../doc/images/output_k8s_deploy-airflow.txt | 2 +- dev/breeze/doc/images/output_k8s_dev.txt | 2 +- dev/breeze/doc/images/output_k8s_k9s.txt | 2 +- .../doc/images/output_k8s_run-complete-tests.txt | 2 +- dev/breeze/doc/images/output_k8s_shell.txt | 2 +- .../doc/images/output_k8s_smoke-test-overlay.txt | 2 +- dev/breeze/doc/images/output_k8s_tests.txt | 2 +- dev/breeze/doc/images/output_prod-image_pull.txt | 2 +- dev/breeze/doc/images/output_prod-image_verify.txt | 2 +- ...tput_release-management_add-back-references.txt | 2 +- ...tput_release-management_check-release-files.txt | 2 +- ...elease-management_classify-provider-changes.txt | 2 +- ...management_generate-issue-content-providers.txt | 2 +- ...e-management_prepare-provider-distributions.txt | 2 +- ...e-management_prepare-provider-documentation.txt | 2 +- .../output_release-management_publish-docs.txt | 2 +- dev/breeze/doc/images/output_run.txt | 2 +- dev/breeze/doc/images/output_shell.txt | 2 +- dev/breeze/doc/images/output_start-airflow.txt | 2 +- ...utput_testing_airflow-ctl-integration-tests.txt | 2 +- .../images/output_testing_airflow-ctl-tests.txt | 2 +- .../images/output_testing_airflow-e2e-tests.txt | 2 +- .../output_testing_core-integration-tests.txt | 2 +- .../doc/images/output_testing_core-tests.txt | 2 +- .../images/output_testing_docker-compose-tests.txt | 2 +- .../doc/images/output_testing_helm-tests.txt | 2 +- .../output_testing_providers-integration-tests.txt | 2 +- .../doc/images/output_testing_providers-tests.txt | 2 +- .../output_testing_python-api-client-tests.txt | 2 +- .../doc/images/output_testing_system-tests.txt | 2 +- .../output_testing_task-sdk-integration-tests.txt | 2 +- .../doc/images/output_testing_task-sdk-tests.txt | 2 +- .../doc/images/output_testing_ui-e2e-tests.txt | 2 +- .../images/output_workflow-run_publish-docs.txt | 2 +- dev/breeze/src/airflow_breeze/utils/path_utils.py | 10 +-- dev/breeze/src/airflow_breeze/utils/reinstall.py | 13 ++-- dev/breeze/tests/test_shim_version_check.py | 8 +-- dev/skill-evals/last-eval-hash.txt | 2 +- scripts/ci/install_breeze.sh | 7 +- scripts/ci/prek/breeze_cmd_line.py | 40 ++++++++++-- scripts/ci/prek/common_prek_utils.py | 55 +++++++++++++++- scripts/tests/ci/prek/test_breeze_cmd_line.py | 10 +-- scripts/tests/ci/prek/test_common_prek_utils.py | 47 ++++++++++++++ scripts/tools/setup_breeze | 16 +++-- 58 files changed, 281 insertions(+), 116 deletions(-) diff --git a/.github/workflows/basic-tests.yml b/.github/workflows/basic-tests.yml index 211d77e48c8..f2f2198c8e6 100644 --- a/.github/workflows/basic-tests.yml +++ b/.github/workflows/basic-tests.yml @@ -95,7 +95,7 @@ jobs: - name: "Install Breeze" uses: ./.github/actions/breeze - name: "Run unit tests" - run: uv tool run --from apache-airflow-breeze pytest -n auto --color=yes + run: uv run --locked pytest -n auto --color=yes working-directory: ./dev/breeze/ run-breeze-integration-tests: timeout-minutes: 120 @@ -122,7 +122,7 @@ jobs: - name: "Install hatch" run: uv tool install hatch - name: "Run integration tests" - run: uv tool run --from apache-airflow-breeze pytest -v --color=yes -m integration_tests + run: uv run --locked pytest -v --color=yes -m integration_tests working-directory: ./dev/breeze/ tests-shared-distributions: timeout-minutes: 10 diff --git a/AGENTS.md b/AGENTS.md index c414057ac1f..46c1ab0f3dd 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -22,7 +22,7 @@ Don't spell out **Directed Acyclic Graph** except for historical context. - Install prek: `uv tool install prek` - Enable commit hooks: `prek install` -- Install breeze shim (one-time, per machine): `scripts/tools/setup_breeze` — installs `~/.local/bin/breeze` that runs breeze via `uvx` from the current git worktree's `dev/breeze` (so each worktree, including ephemeral agent worktrees, gets its own breeze tied to its sources). See [ADR 0017](dev/breeze/doc/adr/0017-use-uvx-to-run-breeze-from-local-sources.md). +- Install breeze shim (one-time, per machine): `scripts/tools/setup_breeze` — installs `~/.local/bin/breeze` that runs breeze via `uv run --locked` from the current git worktree's `dev/breeze`, with dependencies pinned by `dev/breeze/uv.lock` (so each worktree, including ephemeral agent worktrees, gets its own breeze tied to its sources). See [ADR 0017](dev/breeze/doc/adr/0017-use-uvx-to-run-breeze-from-local-sources.md). - **Never run pytest, python, or airflow commands directly on the host** — always use `breeze`. - Place temporary scripts in `dev/` (mounted as `/opt/airflow/dev/` inside Breeze). diff --git a/contributing-docs/03_contributors_quick_start.rst b/contributing-docs/03_contributors_quick_start.rst index f08fef2deec..e0ac6c59b47 100644 --- a/contributing-docs/03_contributors_quick_start.rst +++ b/contributing-docs/03_contributors_quick_start.rst @@ -414,9 +414,10 @@ see in CI in your local environment. ``uv`` is the recommended general-purpose Python development environment for Airflow. 2. Run ``./scripts/tools/setup_breeze`` in your checked-out repository. This installs a small shim - at ``~/.local/bin/breeze`` that runs Breeze via ``uvx`` from the current git worktree's - ``dev/breeze`` folder, so each worktree (including ephemeral ones used by coding agents) gets - its own Breeze tied to that worktree's sources. See + at ``~/.local/bin/breeze`` that runs Breeze via ``uv run --locked`` from the current git + worktree's ``dev/breeze`` folder, so each worktree (including ephemeral ones used by coding + agents) gets its own Breeze, tied to that worktree's sources and to the dependency versions + pinned in ``dev/breeze/uv.lock``. See `ADR 0017 <../dev/breeze/doc/adr/0017-use-uvx-to-run-breeze-from-local-sources.md>`_ for the rationale. diff --git a/contributing-docs/quick-start-ide/contributors_quick_start_gitpod.rst b/contributing-docs/quick-start-ide/contributors_quick_start_gitpod.rst index 5f8694cbeec..b3e49802a65 100644 --- a/contributing-docs/quick-start-ide/contributors_quick_start_gitpod.rst +++ b/contributing-docs/quick-start-ide/contributors_quick_start_gitpod.rst @@ -65,8 +65,9 @@ the shim installer, which works the same in Gitpod as on a local machine: pip install uv ./scripts/tools/setup_breeze -This installs ``~/.local/bin/breeze`` as a small shim that runs Breeze via ``uvx`` from the -current git worktree's ``dev/breeze`` folder. See +This installs ``~/.local/bin/breeze`` as a small shim that runs Breeze via ``uv run --locked`` +from the current git worktree's ``dev/breeze`` folder, with dependencies pinned by +``dev/breeze/uv.lock``. See `ADR 0017 <../../dev/breeze/doc/adr/0017-use-uvx-to-run-breeze-from-local-sources.md>`_ for the rationale. diff --git a/dev/README_AIRFLOW3_DEV.md b/dev/README_AIRFLOW3_DEV.md index a947d5d01cc..d99a81a60f7 100644 --- a/dev/README_AIRFLOW3_DEV.md +++ b/dev/README_AIRFLOW3_DEV.md @@ -130,7 +130,8 @@ To test your changes locally, check out the `v2-11-test` branch. Breeze on Airfl not compatible with Airflow 2.11, so you need a Breeze that matches the branch you're on. If you installed Breeze via the recommended shim (`./scripts/tools/setup_breeze`), nothing extra -is needed — the shim runs Breeze via `uvx` from the current git worktree's `dev/breeze`, so +is needed — the shim runs Breeze via `uv run --locked` from the current git worktree's +`dev/breeze`, so checking out a different branch (or using a separate git worktree) automatically picks up that branch's Breeze: diff --git a/dev/README_RELEASE_AIRFLOW.md b/dev/README_RELEASE_AIRFLOW.md index 6f140f3126e..3fd3cf6e89f 100644 --- a/dev/README_RELEASE_AIRFLOW.md +++ b/dev/README_RELEASE_AIRFLOW.md @@ -388,7 +388,7 @@ export AIRFLOW_REPO_ROOT=$(pwd) ``` - Install `breeze` command (recommended — installs a shim at `~/.local/bin/breeze` that runs - breeze via `uvx` from the current git worktree's `dev/breeze`; see + breeze via `uv run --locked` from the current git worktree's `dev/breeze`; see [ADR 0017](breeze/doc/adr/0017-use-uvx-to-run-breeze-from-local-sources.md)): ```shell script diff --git a/dev/breeze/README.md b/dev/breeze/README.md index f7cc35c1d59..992be53f91c 100644 --- a/dev/breeze/README.md +++ b/dev/breeze/README.md @@ -40,7 +40,8 @@ out, in [editable/development mode](https://packaging.python.org/en/latest/guide The recommended way to make `breeze` available is to install a small **shim script** at `~/.local/bin/breeze` that runs breeze from the `dev/breeze` folder of the *current* git -worktree via `uvx`. This avoids a single global install and means each git worktree +worktree via `uv run --locked`, with every dependency pinned by the committed +`dev/breeze/uv.lock`. This avoids a single global install and means each git worktree (including ephemeral worktrees used by coding agents) gets its own breeze, tied to that worktree's sources. Because the shim is a real file on `PATH`, subprocesses (pre-commit hooks, CI scripts, dev tools) see it just like a `uv tool`-installed binary. See @@ -84,12 +85,12 @@ else exit 1 fi exec env AIRFLOW_ROOT_PATH="${breeze_root}" SKIP_BREEZE_SELF_UPGRADE_CHECK=1 \ - uvx --from "${breeze_root}/dev/breeze" --quiet breeze "$@" + uv run --project "${breeze_root}/dev/breeze" --locked --quiet breeze "$@" ``` Then `breeze` invoked from any Airflow checkout uses that checkout's source, and from anywhere else it uses `$AIRFLOW_REPO_ROOT` or the baked-in fallback. The first call in a -fresh worktree pays a one-time `uvx` resolve/install; subsequent calls hit the cache. +fresh worktree pays a one-time sync of `dev/breeze/.venv`; subsequent calls reuse it. The legacy global-install path (`uv tool install -e ./dev/breeze --force` or `pipx install -e ./dev/breeze --force`) still works for users who explicitly want a single diff --git a/dev/breeze/doc/01_installation.rst b/dev/breeze/doc/01_installation.rst index eb337d29aa4..c54a5350768 100644 --- a/dev/breeze/doc/01_installation.rst +++ b/dev/breeze/doc/01_installation.rst @@ -269,7 +269,8 @@ Set your working directory to the root of this cloned repository. The recommended way to make ``breeze`` available is to install a small **shim script** at ``~/.local/bin/breeze`` that runs breeze from the ``dev/breeze`` folder of the current git -worktree via ``uvx``. This avoids a single global install and means each git worktree +worktree via ``uv run --locked``, with every dependency pinned by the committed +``dev/breeze/uv.lock``. This avoids a single global install and means each git worktree (including ephemeral worktrees used by coding agents) gets its own breeze, tied to that worktree's sources. Because the shim is a real file on ``PATH``, subprocesses (pre-commit hooks, CI scripts, dev tools) see it just like a ``uv tool``-installed binary. See @@ -300,10 +301,10 @@ marks it executable. To do it manually, write this file to ``~/.local/bin/breeze exit 1 fi exec env AIRFLOW_ROOT_PATH="${repo_root}" SKIP_BREEZE_SELF_UPGRADE_CHECK=1 \ - uvx --from "${repo_root}/dev/breeze" --quiet breeze "$@" + uv run --project "${repo_root}/dev/breeze" --locked --quiet breeze "$@" Then ``breeze`` invoked from any Airflow checkout uses that checkout's source. The first call in -a fresh worktree pays a one-time ``uvx`` resolve/install; subsequent calls hit the cache. +a fresh worktree pays a one-time sync of ``dev/breeze/.venv``; subsequent calls reuse it. Alternative: legacy global install (``uv tool`` or ``pipx``) ------------------------------------------------------------ diff --git a/dev/breeze/doc/adr/0016-use-uv-tool-to-install-breeze.md b/dev/breeze/doc/adr/0016-use-uv-tool-to-install-breeze.md index 21e21b335c1..696bad76987 100644 --- a/dev/breeze/doc/adr/0016-use-uv-tool-to-install-breeze.md +++ b/dev/breeze/doc/adr/0016-use-uv-tool-to-install-breeze.md @@ -35,7 +35,7 @@ Date: 2024-11-11 ## Status -Superseded by [17. Use `uvx` to run breeze from local sources](0017-use-uvx-to-run-breeze-from-local-sources.md) +Superseded by [17. Run breeze from the current worktree's locked sources](0017-use-uvx-to-run-breeze-from-local-sources.md) Supersedes [10. Use pipx to install breeze](0010-use-pipx-to-install-breeze.md) diff --git a/dev/breeze/doc/adr/0017-use-uvx-to-run-breeze-from-local-sources.md b/dev/breeze/doc/adr/0017-use-uvx-to-run-breeze-from-local-sources.md index 017fd6d1d18..8f2a06e2071 100644 --- a/dev/breeze/doc/adr/0017-use-uvx-to-run-breeze-from-local-sources.md +++ b/dev/breeze/doc/adr/0017-use-uvx-to-run-breeze-from-local-sources.md @@ -21,7 +21,7 @@ <!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE --> **Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)* -- [17. Use `uvx` to run breeze from local sources](#17-use-uvx-to-run-breeze-from-local-sources) +- [17. Run breeze from the current worktree's locked sources](#17-run-breeze-from-the-current-worktrees-locked-sources) - [Status](#status) - [Context](#context) - [Decision](#decision) @@ -29,9 +29,9 @@ <!-- END doctoc generated TOC please keep comment here to allow auto update --> -# 17. Use `uvx` to run breeze from local sources +# 17. Run breeze from the current worktree's locked sources -Date: 2026-04-26 +Date: 2026-04-26 (amended 2026-08-26: dispatch moved from ``uvx`` to ``uv run --locked``) ## Status @@ -65,12 +65,23 @@ Two patterns have made that single-install model awkward: the same ``~/.local/bin/breeze`` symlink, and an agent that does ``uv tool install --force`` to "fix" itself silently sabotages every other worktree on the machine. -``uv`` ships a tool — ``uvx`` — that runs a command from a project directory in an -ephemeral, cached environment without installing anything globally. ``uvx --from -./dev/breeze breeze ...`` resolves dependencies once per ``pyproject.toml`` /``uv.lock`` -hash, caches the resulting environment, and reuses it on subsequent calls. The first -call in a fresh worktree is slow (one resolve + install); every call after that is -fast. +``uv`` ships a way to run a command from a project directory without installing anything +globally: ``uv run --project ./dev/breeze --locked breeze ...`` syncs that project's own +environment (``dev/breeze/.venv``) to exactly what ``dev/breeze/uv.lock`` pins, then runs the +command in it. The first call in a fresh worktree pays for the sync; every call after that +reuses the environment. + +The ``--locked`` part matters as much as the per-worktree part. The two alternatives that +install from a path — ``uvx --from ./dev/breeze`` and ``uv tool install -e ./dev/breeze`` — +re-resolve breeze's requirements against the package index on every fresh environment and read +neither ``dev/breeze/uv.lock`` nor the ``[tool.uv] exclude-newer`` buffer declared next to it +(that setting governs project operations such as ``uv lock`` and ``uv sync`` only). Under those, +the committed lock recorded nothing and an unrelated upstream release could change breeze with +no commit in this repository. click 8.5.0 showed the cost: released on 2026-08-26, it added a +``help`` field to ``click.Argument.to_info_dict()`` — the dict breeze hashes to detect command +drift — so within the hour every CI job resolved the new click, every command taking a +positional argument hashed differently from its committed value, and static checks went red on +every open PR regardless of what it touched. The lock said click 8.4.2 throughout. That gives us a way to make ``breeze`` always run from the *current* worktree's source without ever touching a shared global install — but the dispatch mechanism has to be @@ -87,9 +98,9 @@ The recommended way to run breeze is via a small **shim script** at ```shell #!/usr/bin/env bash # Apache Airflow breeze shim — managed by scripts/tools/setup_breeze (ADR 0017). -# Runs breeze from the dev/breeze folder of the current git worktree via 'uvx', -# so each worktree (e.g. parallel agentic runs) gets its own ephemerally-installed -# breeze tied to that worktree's source. +# Runs breeze from the dev/breeze folder of the current git worktree via 'uv run', +# so each worktree (e.g. parallel agentic runs) gets its own environment tied to +# that worktree's source, with dependencies resolved from dev/breeze/uv.lock. # # Resolution order for the Airflow sources breeze runs from: # 1. the current git worktree (per-worktree isolation — see above); @@ -115,7 +126,7 @@ else exit 1 fi exec env AIRFLOW_ROOT_PATH="${breeze_root}" SKIP_BREEZE_SELF_UPGRADE_CHECK=1 \ - uvx --from "${breeze_root}/dev/breeze" --quiet breeze "$@" + uv run --project "${breeze_root}/dev/breeze" --locked --quiet breeze "$@" ``` ``scripts/tools/setup_breeze`` writes this file (replacing any previous @@ -128,8 +139,9 @@ The user-facing command stays the same — they still type ``breeze`` — but ea invocation: * resolves ``$(git rev-parse --show-toplevel)`` from the current working directory, -* dispatches to ``uvx --from <that-worktree>/dev/breeze breeze``, -* and therefore always runs the breeze code that belongs to that worktree. +* dispatches to ``uv run --project <that-worktree>/dev/breeze --locked breeze``, +* and therefore always runs the breeze code that belongs to that worktree, with the + dependencies that worktree's ``uv.lock`` pins. Because the shim is a real file on ``PATH`` (not a shell function), it is also visible to subprocesses — pre-commit hooks, CI scripts, dev tools, and anything @@ -137,11 +149,17 @@ else that does ``subprocess.run(["breeze", ...])`` will pick it up exactly like they picked up the old ``uv tool``-installed binary. The two ``env`` variables matter: ``AIRFLOW_ROOT_PATH`` short-circuits breeze's -installation-source detection (which walks up from ``__file__`` and would -otherwise misfire because ``__file__`` lives inside the uvx cache, not the -source tree), and ``SKIP_BREEZE_SELF_UPGRADE_CHECK=1`` disables the "your -install is older than your sources" nag — moot under uvx, which auto-rebuilds -the env when ``pyproject.toml`` / ``uv.lock`` change. +installation-source detection (which walks up from ``__file__`` and would otherwise +misfire on installs that do not live in the source tree), and +``SKIP_BREEZE_SELF_UPGRADE_CHECK=1`` disables the "your install is older than your +sources" nag — moot here, since ``uv run`` re-syncs the environment whenever +``pyproject.toml`` / ``uv.lock`` change and installs the sources as editable. + +CI installs breeze the same way: ``scripts/ci/install_breeze.sh`` runs +``uv sync --project ./dev/breeze/ --locked`` and puts ``dev/breeze/.venv/bin`` on ``PATH`` +rather than installing a global ``uv tool``. Dependency upgrades reach breeze only through a +change to ``dev/breeze/uv.lock`` — in practice the scheduled ``breeze ci upgrade`` PR, which +regenerates the lock and the command-output files together, as one reviewable commit. ``uv tool install -e ./dev/breeze`` and ``pipx install -e ./dev/breeze`` remain supported as alternatives for users who explicitly want the old single-install @@ -159,6 +177,10 @@ behaviour, but they are no longer the recommended path. checked out — not whatever was current the last time someone reinstalled. The "your installed breeze is older than your sources" warning class largely goes away. +* **Reproducible dependencies.** Two checkouts of the same commit run breeze with the + same dependency versions, whatever the index served that day, so the command hashes + under ``dev/breeze/doc/images/`` are a property of the repository rather than of the + calendar — and the ``exclude-newer`` buffer around lock upgrades finally applies. * **Cheap setup in fresh worktrees.** Spinning up a new worktree (manually or via an agent) needs no extra install step; ``breeze`` works the moment ``cd`` lands in the tree. @@ -175,11 +197,15 @@ behaviour, but they are no longer the recommended path. **Costs** -* **First call in a new worktree is slow.** ``uvx`` has to resolve and install - breeze's dependencies the first time it sees a given ``pyproject.toml`` / - ``uv.lock``. Subsequent calls hit the cache and are fast. +* **First call in a new worktree is slow.** ``uv run`` has to populate + ``dev/breeze/.venv`` (~275 MB, mostly hardlinked into the uv cache; ignored by both + ``.gitignore`` and ``.dockerignore``) the first time. Subsequent calls reuse it. +* **A stale lock blocks breeze.** Editing ``dev/breeze/pyproject.toml`` without re-running + ``uv lock`` makes every breeze call fail until the lock is refreshed. The error names the + fix, and the alternative — silently running dependencies nobody recorded — is the failure + mode this dispatch removes. * **Adds a small bash startup overhead.** The shim is a tiny bash script that - runs ``git rev-parse`` and ``uvx`` for every invocation. Negligible at the + runs ``git rev-parse`` and ``uv run`` for every invocation. Negligible at the command line, but noticeable inside tight loops or shell completion that re-invokes ``breeze`` many times. * **Resolution is current-worktree-first, with two fallbacks.** ``breeze`` diff --git a/dev/breeze/doc/images/output_build-docs.txt b/dev/breeze/doc/images/output_build-docs.txt index c532c1df7a8..f2015f69a55 100644 --- a/dev/breeze/doc/images/output_build-docs.txt +++ b/dev/breeze/doc/images/output_build-docs.txt @@ -1 +1 @@ -43e56152396620517257cbcd0ce28ccd +f941b35af1d912596b27d89d4863dd0a diff --git a/dev/breeze/doc/images/output_ci-image_pull.txt b/dev/breeze/doc/images/output_ci-image_pull.txt index 8c10da62eff..fb2ef908512 100644 --- a/dev/breeze/doc/images/output_ci-image_pull.txt +++ b/dev/breeze/doc/images/output_ci-image_pull.txt @@ -1 +1 @@ -d23c74c8aa244d8b8cd55c9182049f10 +fc42909eb684d7821ff2a3575a7e1073 diff --git a/dev/breeze/doc/images/output_ci-image_verify.txt b/dev/breeze/doc/images/output_ci-image_verify.txt index 874ad0f2991..d62405260cc 100644 --- a/dev/breeze/doc/images/output_ci-image_verify.txt +++ b/dev/breeze/doc/images/output_ci-image_verify.txt @@ -1 +1 @@ -3f40148d24cfcc454145bc9971902ead +59daf613dc467f3f1f923bcf7d24eb88 diff --git a/dev/breeze/doc/images/output_exec.txt b/dev/breeze/doc/images/output_exec.txt index a7708253212..b3ad835922b 100644 --- a/dev/breeze/doc/images/output_exec.txt +++ b/dev/breeze/doc/images/output_exec.txt @@ -1 +1 @@ -08e27060ce2c204f4a2b5b1d0c326c74 +0c19dd1538b155da498dd37640c9327e diff --git a/dev/breeze/doc/images/output_k8s_deploy-airflow.txt b/dev/breeze/doc/images/output_k8s_deploy-airflow.txt index fc6808f34b1..f4c4c5620af 100644 --- a/dev/breeze/doc/images/output_k8s_deploy-airflow.txt +++ b/dev/breeze/doc/images/output_k8s_deploy-airflow.txt @@ -1 +1 @@ -bcbd14d163c29520124a0049a1b084dd +3ce1a62e91cffceefefd09619cb6eb52 diff --git a/dev/breeze/doc/images/output_k8s_dev.txt b/dev/breeze/doc/images/output_k8s_dev.txt index 2269c7f1dfc..fc220faba97 100644 --- a/dev/breeze/doc/images/output_k8s_dev.txt +++ b/dev/breeze/doc/images/output_k8s_dev.txt @@ -1 +1 @@ -ba04bbdd68c6b991e906625178b3478f +012f58c236bd03a3dd0a1578af2a4d85 diff --git a/dev/breeze/doc/images/output_k8s_k9s.txt b/dev/breeze/doc/images/output_k8s_k9s.txt index a0b5531337c..eb49e665e92 100644 --- a/dev/breeze/doc/images/output_k8s_k9s.txt +++ b/dev/breeze/doc/images/output_k8s_k9s.txt @@ -1 +1 @@ -83f4fffcd60ffdf1711983ffa43a5eee +81eeaa4e9ab9ab7ec358057d6e4e0968 diff --git a/dev/breeze/doc/images/output_k8s_run-complete-tests.txt b/dev/breeze/doc/images/output_k8s_run-complete-tests.txt index 971ed6e834e..932dbefc44e 100644 --- a/dev/breeze/doc/images/output_k8s_run-complete-tests.txt +++ b/dev/breeze/doc/images/output_k8s_run-complete-tests.txt @@ -1 +1 @@ -abc593ccc646cf825fab808db673a269 +bbff103f342a54b5f22b0a3577d3a632 diff --git a/dev/breeze/doc/images/output_k8s_shell.txt b/dev/breeze/doc/images/output_k8s_shell.txt index d4eaa5a3ee3..84c5b435e6f 100644 --- a/dev/breeze/doc/images/output_k8s_shell.txt +++ b/dev/breeze/doc/images/output_k8s_shell.txt @@ -1 +1 @@ -f75f1f3af7d1bc1e4b7cdd5fd1747ee9 +2b3987215e9ca3cf742199fba144c0e2 diff --git a/dev/breeze/doc/images/output_k8s_smoke-test-overlay.txt b/dev/breeze/doc/images/output_k8s_smoke-test-overlay.txt index 087918e8b3d..fb2b987eceb 100644 --- a/dev/breeze/doc/images/output_k8s_smoke-test-overlay.txt +++ b/dev/breeze/doc/images/output_k8s_smoke-test-overlay.txt @@ -1 +1 @@ -912bce7922f7ce90391f7af648ac4636 +c9fae60d9f4e72e390a08d53366d20f9 diff --git a/dev/breeze/doc/images/output_k8s_tests.txt b/dev/breeze/doc/images/output_k8s_tests.txt index 4fcde22a9dd..459e3cd1d2d 100644 --- a/dev/breeze/doc/images/output_k8s_tests.txt +++ b/dev/breeze/doc/images/output_k8s_tests.txt @@ -1 +1 @@ -737cf1595fa7afd91a23ef79f46eff16 +7a1170264859cd3fc078681755907622 diff --git a/dev/breeze/doc/images/output_prod-image_pull.txt b/dev/breeze/doc/images/output_prod-image_pull.txt index b55181ed864..fa188f44b61 100644 --- a/dev/breeze/doc/images/output_prod-image_pull.txt +++ b/dev/breeze/doc/images/output_prod-image_pull.txt @@ -1 +1 @@ -d7ed9abca3180bd5a1648f95a2624413 +943b61b60f81476638e74ab0e5ba451e diff --git a/dev/breeze/doc/images/output_prod-image_verify.txt b/dev/breeze/doc/images/output_prod-image_verify.txt index f124b84cb84..f431e6eae9d 100644 --- a/dev/breeze/doc/images/output_prod-image_verify.txt +++ b/dev/breeze/doc/images/output_prod-image_verify.txt @@ -1 +1 @@ -a2f42c9ae4c6b66967588bd5ab538031 +f95887aff9ae25fe203629e01c2fe25d diff --git a/dev/breeze/doc/images/output_release-management_add-back-references.txt b/dev/breeze/doc/images/output_release-management_add-back-references.txt index 4cdc1d4f62f..ad57d752e8b 100644 --- a/dev/breeze/doc/images/output_release-management_add-back-references.txt +++ b/dev/breeze/doc/images/output_release-management_add-back-references.txt @@ -1 +1 @@ -3162d4ae447d02bdddf9d67590fb4134 +bc1e2c3d12063d93fa3f3605b4f42de1 diff --git a/dev/breeze/doc/images/output_release-management_check-release-files.txt b/dev/breeze/doc/images/output_release-management_check-release-files.txt index 2a16aae00bf..e3b08d31a5e 100644 --- a/dev/breeze/doc/images/output_release-management_check-release-files.txt +++ b/dev/breeze/doc/images/output_release-management_check-release-files.txt @@ -1 +1 @@ -465b1fffb9a13eb9a97ffa2dcadd4c58 +4d7d540413a5955a51eaf7d035ff31bc diff --git a/dev/breeze/doc/images/output_release-management_classify-provider-changes.txt b/dev/breeze/doc/images/output_release-management_classify-provider-changes.txt index a78f36b3ac3..44b287551c6 100644 --- a/dev/breeze/doc/images/output_release-management_classify-provider-changes.txt +++ b/dev/breeze/doc/images/output_release-management_classify-provider-changes.txt @@ -1 +1 @@ -ce0ed1333a3f38983043d4eb04f0bf1c +0f0f937f1a0d86a58147723b1400b3ac diff --git a/dev/breeze/doc/images/output_release-management_generate-issue-content-providers.txt b/dev/breeze/doc/images/output_release-management_generate-issue-content-providers.txt index a249fc1a2ed..91f9d9e152d 100644 --- a/dev/breeze/doc/images/output_release-management_generate-issue-content-providers.txt +++ b/dev/breeze/doc/images/output_release-management_generate-issue-content-providers.txt @@ -1 +1 @@ -ec8538954daf8fe21156a7ad72992e85 +465c7719885f58cdf5fc8bd0592ba7de diff --git a/dev/breeze/doc/images/output_release-management_prepare-provider-distributions.txt b/dev/breeze/doc/images/output_release-management_prepare-provider-distributions.txt index 5203efb313d..4d4703a71ae 100644 --- a/dev/breeze/doc/images/output_release-management_prepare-provider-distributions.txt +++ b/dev/breeze/doc/images/output_release-management_prepare-provider-distributions.txt @@ -1 +1 @@ -b635f3eb5c8e04be0937b9e85dee856a +b937b6a253ce2ab2cbabb9cd8b7369d4 diff --git a/dev/breeze/doc/images/output_release-management_prepare-provider-documentation.txt b/dev/breeze/doc/images/output_release-management_prepare-provider-documentation.txt index e7ca97a104b..5b7184cf9c6 100644 --- a/dev/breeze/doc/images/output_release-management_prepare-provider-documentation.txt +++ b/dev/breeze/doc/images/output_release-management_prepare-provider-documentation.txt @@ -1 +1 @@ -a94ab59178e740921f451bdc68bfb445 +57343f19b12400ac43ea8fc4b892ac99 diff --git a/dev/breeze/doc/images/output_release-management_publish-docs.txt b/dev/breeze/doc/images/output_release-management_publish-docs.txt index ba3f69ebcb5..2e522c29cf7 100644 --- a/dev/breeze/doc/images/output_release-management_publish-docs.txt +++ b/dev/breeze/doc/images/output_release-management_publish-docs.txt @@ -1 +1 @@ -f08a2861a8c88397caa6ea8471819456 +1a68af03bc2f8fae0d46e4a2a2266eb0 diff --git a/dev/breeze/doc/images/output_run.txt b/dev/breeze/doc/images/output_run.txt index 4a274308775..c107633ee5a 100644 --- a/dev/breeze/doc/images/output_run.txt +++ b/dev/breeze/doc/images/output_run.txt @@ -1 +1 @@ -e850a14ddf66f700b2595d65d7511869 +d4841617ec2b8d96643f6695adcb68d6 diff --git a/dev/breeze/doc/images/output_shell.txt b/dev/breeze/doc/images/output_shell.txt index 8e1ee65300e..796f1d521e1 100644 --- a/dev/breeze/doc/images/output_shell.txt +++ b/dev/breeze/doc/images/output_shell.txt @@ -1 +1 @@ -0278a7f8d4d43cf91efbaaae1368326e +cfa0588bb3e466155cd793b4528ac28c diff --git a/dev/breeze/doc/images/output_start-airflow.txt b/dev/breeze/doc/images/output_start-airflow.txt index fa1d44f74c7..e2dc6a5a11e 100644 --- a/dev/breeze/doc/images/output_start-airflow.txt +++ b/dev/breeze/doc/images/output_start-airflow.txt @@ -1 +1 @@ -b4b01862a54de30d7df835ee6f9150df +3cdf6229fb227fdbf3f7b0524b4a7f75 diff --git a/dev/breeze/doc/images/output_testing_airflow-ctl-integration-tests.txt b/dev/breeze/doc/images/output_testing_airflow-ctl-integration-tests.txt index 446ec42dd1f..36147a39ccc 100644 --- a/dev/breeze/doc/images/output_testing_airflow-ctl-integration-tests.txt +++ b/dev/breeze/doc/images/output_testing_airflow-ctl-integration-tests.txt @@ -1 +1 @@ -4c341666e70a017acd38edd879adb595 +e302335890458a12a28a52ed51b6337f diff --git a/dev/breeze/doc/images/output_testing_airflow-ctl-tests.txt b/dev/breeze/doc/images/output_testing_airflow-ctl-tests.txt index 5799a6650b8..98f03c94d23 100644 --- a/dev/breeze/doc/images/output_testing_airflow-ctl-tests.txt +++ b/dev/breeze/doc/images/output_testing_airflow-ctl-tests.txt @@ -1 +1 @@ -f11826c57955e3a6bef2f3c95989cacc +caf04a20ceb4b3a426e095860f8a800f diff --git a/dev/breeze/doc/images/output_testing_airflow-e2e-tests.txt b/dev/breeze/doc/images/output_testing_airflow-e2e-tests.txt index d2467d70674..31336273b3b 100644 --- a/dev/breeze/doc/images/output_testing_airflow-e2e-tests.txt +++ b/dev/breeze/doc/images/output_testing_airflow-e2e-tests.txt @@ -1 +1 @@ -2bbd90928e950df8cf78abd0aff50f0b +3005f4343ce3d9c33960042d8d7e2d20 diff --git a/dev/breeze/doc/images/output_testing_core-integration-tests.txt b/dev/breeze/doc/images/output_testing_core-integration-tests.txt index cde65e2f90c..aad3803373c 100644 --- a/dev/breeze/doc/images/output_testing_core-integration-tests.txt +++ b/dev/breeze/doc/images/output_testing_core-integration-tests.txt @@ -1 +1 @@ -ccda221c325794a1a29ff0b77159dbeb +56f677845a9c84dbdc1131c51b9af83a diff --git a/dev/breeze/doc/images/output_testing_core-tests.txt b/dev/breeze/doc/images/output_testing_core-tests.txt index a4246da8dce..ce17b293f36 100644 --- a/dev/breeze/doc/images/output_testing_core-tests.txt +++ b/dev/breeze/doc/images/output_testing_core-tests.txt @@ -1 +1 @@ -2d18c088667df831820438a03ba9654a +e89451fe0b493819e12e78316a070cee diff --git a/dev/breeze/doc/images/output_testing_docker-compose-tests.txt b/dev/breeze/doc/images/output_testing_docker-compose-tests.txt index 46a207d8796..2c906c51d5d 100644 --- a/dev/breeze/doc/images/output_testing_docker-compose-tests.txt +++ b/dev/breeze/doc/images/output_testing_docker-compose-tests.txt @@ -1 +1 @@ -04af62df42438f286d861f3f1956e5e4 +e7f68555c25e802df53097e5e7558088 diff --git a/dev/breeze/doc/images/output_testing_helm-tests.txt b/dev/breeze/doc/images/output_testing_helm-tests.txt index fb44f419c4d..5331937a401 100644 --- a/dev/breeze/doc/images/output_testing_helm-tests.txt +++ b/dev/breeze/doc/images/output_testing_helm-tests.txt @@ -1 +1 @@ -43eb5774bbdef528e443613d1d8048d9 +02ba09ea9213ae046fe814563630054d diff --git a/dev/breeze/doc/images/output_testing_providers-integration-tests.txt b/dev/breeze/doc/images/output_testing_providers-integration-tests.txt index 8626377a020..da0296790d8 100644 --- a/dev/breeze/doc/images/output_testing_providers-integration-tests.txt +++ b/dev/breeze/doc/images/output_testing_providers-integration-tests.txt @@ -1 +1 @@ -fc0029baa8c28609f1e6972829465401 +1ab3cf1b8c5a5deb4181110bb41fb811 diff --git a/dev/breeze/doc/images/output_testing_providers-tests.txt b/dev/breeze/doc/images/output_testing_providers-tests.txt index 770a5cb7598..4efab5ed281 100644 --- a/dev/breeze/doc/images/output_testing_providers-tests.txt +++ b/dev/breeze/doc/images/output_testing_providers-tests.txt @@ -1 +1 @@ -776d6d9819a1351ff4f96f3884f9e039 +9372d4ac91f1f8aec84665bd5bbc7dc6 diff --git a/dev/breeze/doc/images/output_testing_python-api-client-tests.txt b/dev/breeze/doc/images/output_testing_python-api-client-tests.txt index 57a62e8cd40..ca9a2d70afa 100644 --- a/dev/breeze/doc/images/output_testing_python-api-client-tests.txt +++ b/dev/breeze/doc/images/output_testing_python-api-client-tests.txt @@ -1 +1 @@ -64c20fc3168ebec7147f51be8e323040 +0d407ea088f01326dfc1104f12a3017d diff --git a/dev/breeze/doc/images/output_testing_system-tests.txt b/dev/breeze/doc/images/output_testing_system-tests.txt index b9ebe1ecd13..ecb78ce4d7f 100644 --- a/dev/breeze/doc/images/output_testing_system-tests.txt +++ b/dev/breeze/doc/images/output_testing_system-tests.txt @@ -1 +1 @@ -3a8636011e5f5d6090c61ec0433575e5 +0cdb967ba346e6663ac663acf599b5ae diff --git a/dev/breeze/doc/images/output_testing_task-sdk-integration-tests.txt b/dev/breeze/doc/images/output_testing_task-sdk-integration-tests.txt index 1f0b6d1bf5b..6e754162d01 100644 --- a/dev/breeze/doc/images/output_testing_task-sdk-integration-tests.txt +++ b/dev/breeze/doc/images/output_testing_task-sdk-integration-tests.txt @@ -1 +1 @@ -4d9236fb254edf6d1f6e8ec815b50223 +76d2a06163a1c3f4dbc54f2e4aa13ade diff --git a/dev/breeze/doc/images/output_testing_task-sdk-tests.txt b/dev/breeze/doc/images/output_testing_task-sdk-tests.txt index 66725084dfd..475499ef8a8 100644 --- a/dev/breeze/doc/images/output_testing_task-sdk-tests.txt +++ b/dev/breeze/doc/images/output_testing_task-sdk-tests.txt @@ -1 +1 @@ -7135479eff2be2409fad7759de789087 +58ecc16666e1b4f5c5572533e7ccc0a7 diff --git a/dev/breeze/doc/images/output_testing_ui-e2e-tests.txt b/dev/breeze/doc/images/output_testing_ui-e2e-tests.txt index 40d9b34fd05..69046ce01eb 100644 --- a/dev/breeze/doc/images/output_testing_ui-e2e-tests.txt +++ b/dev/breeze/doc/images/output_testing_ui-e2e-tests.txt @@ -1 +1 @@ -07feb53b4eeacb942d45ad1a70dc8740 +109d8450ccc1fa74ebc2547ae2db41d5 diff --git a/dev/breeze/doc/images/output_workflow-run_publish-docs.txt b/dev/breeze/doc/images/output_workflow-run_publish-docs.txt index 9dd84999bbf..bd1e1fde137 100644 --- a/dev/breeze/doc/images/output_workflow-run_publish-docs.txt +++ b/dev/breeze/doc/images/output_workflow-run_publish-docs.txt @@ -1 +1 @@ -65b7b00a5792d185aee3628fdc2227de +c4e8633fee01c127eb062f8ccfac1c5c diff --git a/dev/breeze/src/airflow_breeze/utils/path_utils.py b/dev/breeze/src/airflow_breeze/utils/path_utils.py index 533e741c61e..9183d1fb96e 100644 --- a/dev/breeze/src/airflow_breeze/utils/path_utils.py +++ b/dev/breeze/src/airflow_breeze/utils/path_utils.py @@ -252,9 +252,9 @@ def warn_if_shim_outdated(airflow_sources: Path, shim_text: str | None = None) - setup_script = airflow_sources / "scripts" / "tools" / "setup_breeze" installed_text = installed_version if installed_version is not None else "unknown (pre-versioning)" console_print( - f"\n[warning]Your breeze shim at {BREEZE_SHIM_PATH} is out of date " + f"\n[warning]Your breeze shim at {BREEZE_SHIM_PATH} needs to be upgraded " f"(installed: {installed_text}, current: {expected_version}).[/]\n" - "[warning]Re-run the setup script to refresh it:[/]\n\n" + "[warning]Re-run the setup script to upgrade it:[/]\n\n" f" {setup_script}\n" ) return True @@ -314,9 +314,9 @@ def warn_if_breeze_launcher_outdated(airflow_sources: Path) -> bool: ) setup_script = airflow_sources / "scripts" / "tools" / "setup_breeze" console_print( - f"\n[warning]Breeze is installed as a legacy global '{legacy}' install, which still works " - "but is no longer the recommended setup (see ADR 0017).[/]\n" - "[warning]Migrate to the per-worktree uvx shim by uninstalling the global install and " + f"\n[warning]Breeze is installed as a legacy global '{legacy}' install, which resolves its " + "dependencies against the package index rather than dev/breeze/uv.lock (see ADR 0017).[/]\n" + "[warning]Migrate to the per-worktree shim by uninstalling the global install and " "running the setup script:[/]\n\n" f" {uninstall_cmd}\n" f" {setup_script}\n" diff --git a/dev/breeze/src/airflow_breeze/utils/reinstall.py b/dev/breeze/src/airflow_breeze/utils/reinstall.py index e6ec61c4cd7..d594bbbce79 100644 --- a/dev/breeze/src/airflow_breeze/utils/reinstall.py +++ b/dev/breeze/src/airflow_breeze/utils/reinstall.py @@ -76,14 +76,15 @@ def reinstall_breeze(breeze_sources: Path, re_run: bool = True): ["pipx", "install", "-e", breeze_sources.as_posix(), "--force"], stderr=subprocess.STDOUT ) else: - # Recommended setup: breeze is invoked via the `uvx`-based shell function - # (see ADR 0017). There is no global install to reinstall — uvx will - # rebuild the cached env on next call when pyproject.toml / uv.lock change. + # Recommended setup: breeze is invoked through the shim (see ADR 0017), which runs + # `uv run --locked` against the worktree. There is no global install to reinstall — + # the next call re-syncs the environment whenever pyproject.toml / uv.lock change. console_print( "[info]No global breeze install detected (uv tool / pipx). " - "Assuming the recommended uvx-based setup — nothing to reinstall.[/]\n" - "[info]If you suspect a stale cached env, clear it with:[/]\n" - " uv cache clean apache-airflow-breeze\n" + "Assuming the recommended shim-based setup — nothing to reinstall.[/]\n" + "[info]If you suspect a broken environment, remove it and let the next call " + "rebuild it:[/]\n" + " rm -rf dev/breeze/.venv\n" ) if re_run: diff --git a/dev/breeze/tests/test_shim_version_check.py b/dev/breeze/tests/test_shim_version_check.py index b027955a1d4..671a1a09689 100644 --- a/dev/breeze/tests/test_shim_version_check.py +++ b/dev/breeze/tests/test_shim_version_check.py @@ -61,7 +61,7 @@ def test_parse_shim_version(version_line, expected): def test_get_expected_shim_version_reads_real_setup_script(): # The real setup_breeze in the sources is the source of truth — keep this test in sync # with the SHIM_VERSION it declares. - assert get_expected_shim_version(ACTUAL_AIRFLOW_SOURCES) == 1 + assert get_expected_shim_version(ACTUAL_AIRFLOW_SOURCES) == 2 def test_get_expected_shim_version_from_fake_sources(tmp_path): @@ -123,7 +123,7 @@ def test_warn_if_shim_outdated_older_installed(tmp_path, monkeypatch, capsys): sources = _fake_sources_with_version(tmp_path, 2) assert warn_if_shim_outdated(sources) is True output = capsys.readouterr().out - assert "out of date" in output + assert "needs to be upgraded" in output assert "setup_breeze" in output @@ -135,7 +135,7 @@ def test_warn_if_shim_outdated_pre_versioning_shim(tmp_path, monkeypatch, capsys sources = _fake_sources_with_version(tmp_path, 1) assert warn_if_shim_outdated(sources) is True output = capsys.readouterr().out - assert "out of date" in output + assert "needs to be upgraded" in output assert "pre-versioning" in output @@ -181,7 +181,7 @@ def test_launcher_check_prefers_shim_version(tmp_path, monkeypatch, capsys): assert warn_if_breeze_launcher_outdated(sources) is True detect.assert_not_called() output = capsys.readouterr().out - assert "out of date" in output + assert "needs to be upgraded" in output @pytest.mark.parametrize( diff --git a/dev/skill-evals/last-eval-hash.txt b/dev/skill-evals/last-eval-hash.txt index 4e72dc42b31..f44ab7e72f8 100644 --- a/dev/skill-evals/last-eval-hash.txt +++ b/dev/skill-evals/last-eval-hash.txt @@ -1,3 +1,3 @@ # Generated by dev/skill-evals/eval.py — do not edit or resolve conflicts by hand. # Run `prek run run-skill-eval --hook-stage manual --all-files` to regenerate. -8cfe725b810c4c2d364d99f63b27ec739545e40032195624c218b0eb23d75859 +f74d85cc733701dd0b39c1024b68868be0b739931613b268dd76a8e0d4d18110 diff --git a/scripts/ci/install_breeze.sh b/scripts/ci/install_breeze.sh index 1f37d0cf32e..088072d627d 100755 --- a/scripts/ci/install_breeze.sh +++ b/scripts/ci/install_breeze.sh @@ -27,7 +27,12 @@ if [[ ${PYTHON_VERSION=} != "" ]]; then fi python -m pip install --upgrade "pip==${PIP_VERSION}" +# A leftover global tool install would shadow the venv script below via ~/.local/bin. uv tool uninstall apache-airflow-breeze >/dev/null 2>&1 || true +# `uv sync --locked` installs exactly what dev/breeze/uv.lock pins. `uv tool install` re-resolved +# against the index instead, so any third-party release landing mid-day silently changed breeze's +# dependencies — and with them the breeze command hashes every PR is checked against. # shellcheck disable=SC2086 -uv tool install ${PYTHON_ARG} --force --editable ./dev/breeze/ +uv sync ${PYTHON_ARG} --project ./dev/breeze/ --locked +echo "$(pwd)/dev/breeze/.venv/bin" >> "${GITHUB_PATH}" echo '/home/runner/.local/bin' >> "${GITHUB_PATH}" diff --git a/scripts/ci/prek/breeze_cmd_line.py b/scripts/ci/prek/breeze_cmd_line.py index 7a694073a42..aa2d3577cd6 100755 --- a/scripts/ci/prek/breeze_cmd_line.py +++ b/scripts/ci/prek/breeze_cmd_line.py @@ -28,7 +28,13 @@ import os import subprocess import sys -from common_prek_utils import AIRFLOW_ROOT_PATH, console, initialize_breeze_prek +from common_prek_utils import ( + AIRFLOW_ROOT_PATH, + SETUP_BREEZE_PATH, + console, + describe_breeze_not_running_from_lock, + initialize_breeze_prek, +) BREEZE_INSTALL_DIR = AIRFLOW_ROOT_PATH / "dev" / "breeze" BREEZE_DOC_DIR = BREEZE_INSTALL_DIR / "doc" @@ -40,12 +46,10 @@ FORCE = os.environ.get("FORCE", "false")[0].lower() == "t" def breeze_env_with_local_sources() -> dict[str, str]: """Return an environment that forces breeze to import the local worktree sources. - The ``breeze`` command on PATH is normally the uvx shim (ADR 0017), which runs - breeze from a *cached* build. That cache does not always reflect uncommitted edits - to ``dev/breeze``: ``uvx --refresh`` / ``--reinstall`` do not rebuild a local path - dependency, only ``uvx --no-cache`` does. When the cache is stale this hook computes - the command hashes / option groups from old code and then either misses a needed - regeneration or *reverts* a correctly regenerated image back to the stale version. + The ``breeze`` command on PATH is not guaranteed to import this worktree's sources: a + legacy global install (``uv tool`` / ``pipx``) stays bound to whichever checkout it was + installed from. Computing the command hashes / option groups from that other checkout's + code either misses a needed regeneration or *reverts* a correctly regenerated image. Prepending the local breeze sources to ``PYTHONPATH`` makes the in-process computation (and the help rendering it spawns) always reflect the current source, @@ -100,12 +104,34 @@ def is_regeneration_needed() -> bool: return result.returncode != 0 +def fail_if_breeze_does_not_run_from_lock() -> None: + """Stop before regenerating anything from a breeze whose dependencies are not the locked ones. + + The stored hashes cover the command definitions *as rendered by the locked click*, so + regenerating from an unlocked install rewrites every one of them and the drift comes back on + the next run from a locked install. + """ + reason = describe_breeze_not_running_from_lock() + if reason is None: + return + console.print( + f"\n[red]Cannot regenerate breeze command output: {reason}.[/]\n\n" + "[yellow]Install or upgrade the breeze shim, which runs breeze from " + "`dev/breeze/uv.lock` (ADR 0017):[/]\n\n" + f" {SETUP_BREEZE_PATH}\n\n" + "[yellow]A legacy global install has to go first — it owns the same path:[/]\n\n" + " uv tool uninstall apache-airflow-breeze # or: pipx uninstall apache-airflow-breeze\n" + ) + sys.exit(1) + + def main() -> int: initialize_breeze_prek(__name__, __file__) return_code = 0 verify_all_commands_described_in_docs() if is_regeneration_needed(): + fail_if_breeze_does_not_run_from_lock() console.print( "\n[bright_blue]Some of the commands changed since last time images were generated. " "Regenerating.\n" diff --git a/scripts/ci/prek/common_prek_utils.py b/scripts/ci/prek/common_prek_utils.py index 40cd8ce5838..3a799b2dd19 100644 --- a/scripts/ci/prek/common_prek_utils.py +++ b/scripts/ci/prek/common_prek_utils.py @@ -346,6 +346,59 @@ def check_uv_version(uv_bin: str = "uv") -> None: sys.exit(1) +BREEZE_SHIM_MARKER = "Apache Airflow breeze shim — managed by scripts/tools/setup_breeze" +BREEZE_SHIM_VERSION_PREFIX = "# breeze-shim-version:" +SETUP_BREEZE_SHIM_VERSION_PREFIX = "SHIM_VERSION=" +SETUP_BREEZE_PATH = AIRFLOW_ROOT_PATH / "scripts" / "tools" / "setup_breeze" +BREEZE_LOCKED_VENV_PATH = AIRFLOW_BREEZE_SOURCES_PATH / ".venv" + + +def _read_shim_version(text: str, prefix: str) -> int | None: + for line in text.splitlines(): + stripped = line.strip() + if stripped.startswith(prefix): + try: + return int(stripped[len(prefix) :].strip().strip("\"'")) + except ValueError: + return None + return None + + +def describe_breeze_not_running_from_lock() -> str | None: + """Describe why the ``breeze`` on PATH does not run from ``dev/breeze/uv.lock``. + + Only the current shim (which dispatches to ``uv run --locked``) and the locked venv CI + syncs run the versions the lock pins. An older shim and a legacy ``uv tool`` / ``pipx`` + install both resolve breeze's dependencies against the index instead, so anything derived + from them — command hashes above all — reflects whatever the index served that day. + + :return: a description of the offending install, or None when breeze runs from the lock. + """ + breeze_bin = shutil.which("breeze") + if breeze_bin is None: + return None + resolved = Path(breeze_bin).resolve() + if resolved.is_relative_to(BREEZE_LOCKED_VENV_PATH.resolve()): + return None + try: + text = Path(breeze_bin).read_text() + except (OSError, UnicodeDecodeError): + return None + if BREEZE_SHIM_MARKER not in text: + return f"`{breeze_bin}` is a legacy global install, which ignores the lock" + expected_version = _read_shim_version(SETUP_BREEZE_PATH.read_text(), SETUP_BREEZE_SHIM_VERSION_PREFIX) + if expected_version is None: + return None + installed_version = _read_shim_version(text, BREEZE_SHIM_VERSION_PREFIX) + if installed_version is not None and installed_version >= expected_version: + return None + installed_text = installed_version if installed_version is not None else "pre-versioning" + return ( + f"the shim at `{breeze_bin}` needs to be upgraded " + f"(installed: {installed_text}, current: {expected_version})" + ) + + def initialize_breeze_prek(name: str, file: str): if name not in ("__main__", "__mp_main__"): raise SystemExit( @@ -363,7 +416,7 @@ def initialize_breeze_prek(name: str, file: str): "[red]The `breeze` command is not on path.[/]\n\n" "[yellow]Please install breeze. Recommended: run `./scripts/tools/setup_breeze` " "from the repo root — it installs a shim at `~/.local/bin/breeze` that runs breeze " - "via `uvx` from the current git worktree (see ADR 0017).\n" + "via `uv run --locked` from the current git worktree (see ADR 0017).\n" "Legacy global install (`uv tool install -e ./dev/breeze` or " "`pipx install -e ./dev/breeze`) still works but is no longer recommended.[/]\n\n" "[bright_blue]You can also set SKIP_BREEZE_PREK_HOOKS env variable to non-empty " diff --git a/scripts/tests/ci/prek/test_breeze_cmd_line.py b/scripts/tests/ci/prek/test_breeze_cmd_line.py index 5e83c718e61..941c3fff1cb 100644 --- a/scripts/tests/ci/prek/test_breeze_cmd_line.py +++ b/scripts/tests/ci/prek/test_breeze_cmd_line.py @@ -24,11 +24,11 @@ from ci.prek.breeze_cmd_line import BREEZE_SOURCES_DIR, breeze_env_with_local_so class TestBreezeEnvWithLocalSources: - """The hook must run breeze against the local worktree sources, not a stale cached build. + """The hook must run breeze against the local worktree sources, not another checkout's. - The ``breeze`` shim (ADR 0017) runs from a uvx cache that does not always reflect - uncommitted ``dev/breeze`` edits. Prepending the local sources to ``PYTHONPATH`` - makes the in-process command-hash / option-group computation use the current code. + A legacy global breeze install stays bound to the checkout it was installed from. + Prepending the local sources to ``PYTHONPATH`` makes the in-process command-hash / + option-group computation use the current worktree's code. """ def test_sets_pythonpath_to_breeze_sources_when_unset(self, monkeypatch): @@ -40,7 +40,7 @@ class TestBreezeEnvWithLocalSources: existing = f"/some/path{os.pathsep}/other/path" monkeypatch.setenv("PYTHONPATH", existing) env = breeze_env_with_local_sources() - # Local breeze sources must come first so they win over the cached build. + # Local breeze sources must come first so they win over any other install. assert env["PYTHONPATH"] == f"{BREEZE_SOURCES_DIR}{os.pathsep}{existing}" assert env["PYTHONPATH"].split(os.pathsep)[0] == str(BREEZE_SOURCES_DIR) diff --git a/scripts/tests/ci/prek/test_common_prek_utils.py b/scripts/tests/ci/prek/test_common_prek_utils.py index b169b9b90dc..9632091cd47 100644 --- a/scripts/tests/ci/prek/test_common_prek_utils.py +++ b/scripts/tests/ci/prek/test_common_prek_utils.py @@ -651,6 +651,53 @@ class TestInitializeBreezePrek: assert exc_info.value.code == 1 +class TestDescribeBreezeNotRunningFromLock: + @staticmethod + def _install(tmp_path, monkeypatch, breeze_body: str, setup_version: str = "2"): + breeze_bin = tmp_path / "breeze" + breeze_bin.write_text(breeze_body) + setup_breeze = tmp_path / "setup_breeze" + setup_breeze.write_text(f'SHIM_VERSION="{setup_version}"\n') + monkeypatch.setattr(common_prek_utils, "SETUP_BREEZE_PATH", setup_breeze) + monkeypatch.setattr(common_prek_utils, "BREEZE_LOCKED_VENV_PATH", tmp_path / "locked" / ".venv") + monkeypatch.setattr(common_prek_utils.shutil, "which", lambda _: str(breeze_bin)) + return breeze_bin + + def _shim(self, version: int | None) -> str: + version_line = f"# breeze-shim-version: {version}\n" if version is not None else "" + return f"#!/usr/bin/env bash\n# {common_prek_utils.BREEZE_SHIM_MARKER}\n{version_line}" + + def test_reports_an_outdated_shim(self, tmp_path, monkeypatch): + self._install(tmp_path, monkeypatch, self._shim(1)) + assert "needs to be upgraded" in common_prek_utils.describe_breeze_not_running_from_lock() + + def test_reports_a_shim_predating_versioning(self, tmp_path, monkeypatch): + self._install(tmp_path, monkeypatch, self._shim(None)) + assert "pre-versioning" in common_prek_utils.describe_breeze_not_running_from_lock() + + def test_reports_a_legacy_global_install(self, tmp_path, monkeypatch): + self._install( + tmp_path, monkeypatch, "#!/usr/bin/env python\nfrom airflow_breeze.breeze import main\n" + ) + assert "legacy global install" in common_prek_utils.describe_breeze_not_running_from_lock() + + def test_accepts_a_current_shim(self, tmp_path, monkeypatch): + self._install(tmp_path, monkeypatch, self._shim(2)) + assert common_prek_utils.describe_breeze_not_running_from_lock() is None + + def test_accepts_the_locked_venv_ci_syncs(self, tmp_path, monkeypatch): + venv_bin = tmp_path / "locked" / ".venv" / "bin" + venv_bin.mkdir(parents=True) + (venv_bin / "breeze").write_text("#!/usr/bin/env python\n") + monkeypatch.setattr(common_prek_utils, "BREEZE_LOCKED_VENV_PATH", tmp_path / "locked" / ".venv") + monkeypatch.setattr(common_prek_utils.shutil, "which", lambda _: str(venv_bin / "breeze")) + assert common_prek_utils.describe_breeze_not_running_from_lock() is None + + def test_accepts_a_missing_breeze(self, monkeypatch): + monkeypatch.setattr(common_prek_utils.shutil, "which", lambda _: None) + assert common_prek_utils.describe_breeze_not_running_from_lock() is None + + class TestTemporaryTscProject: def test_creates_temp_tsconfig(self, tmp_path): tsconfig = tmp_path / "tsconfig.json" diff --git a/scripts/tools/setup_breeze b/scripts/tools/setup_breeze index 45742b43a97..3d946a3f071 100755 --- a/scripts/tools/setup_breeze +++ b/scripts/tools/setup_breeze @@ -41,11 +41,13 @@ SHIM_MARKER="# Apache Airflow breeze shim — managed by scripts/tools/setup_bre # into the installed shim; if the installed shim is older it warns the user to # re-run this script. See warn_if_shim_outdated() in # dev/breeze/src/airflow_breeze/utils/path_utils.py. -SHIM_VERSION="1" +SHIM_VERSION="2" -# The shim itself. Runs breeze via 'uvx' against the dev/breeze folder of the +# The shim itself. Runs breeze via 'uv run' against the dev/breeze folder of the # *current* git worktree, so multiple checkouts / agentic worktrees never -# share a single global install. See ADR 0017. +# share a single global install. '--locked' pins every dependency to the +# committed dev/breeze/uv.lock, so a fresh third-party release cannot change +# what breeze runs with until that lock is upgraded. See ADR 0017. # # When invoked outside any Airflow worktree (e.g. from an SVN release checkout # such as asf-dist during a provider release), the shim falls back to, in order: @@ -55,9 +57,9 @@ read -r -d '' BREEZE_SHIM_BODY <<BREEZE_SHIM || true #!/usr/bin/env bash ${SHIM_MARKER} # breeze-shim-version: ${SHIM_VERSION} -# Runs breeze from the dev/breeze folder of the current git worktree via 'uvx', -# so each worktree (e.g. parallel agentic runs) gets its own ephemerally-installed -# breeze tied to that worktree's source. +# Runs breeze from the dev/breeze folder of the current git worktree via 'uv run', +# so each worktree (e.g. parallel agentic runs) gets its own environment tied to +# that worktree's source, with dependencies resolved from dev/breeze/uv.lock. # # Resolution order for the Airflow sources breeze runs from: # 1. the current git worktree (per-worktree isolation — see ADR 0017); @@ -83,7 +85,7 @@ else exit 1 fi exec env AIRFLOW_ROOT_PATH="\${breeze_root}" SKIP_BREEZE_SELF_UPGRADE_CHECK=1 \\ - uvx --from "\${breeze_root}/dev/breeze" --quiet breeze "\$@" + uv run --project "\${breeze_root}/dev/breeze" --locked --quiet breeze "\$@" BREEZE_SHIM function manual_instructions() {
