kaxil opened a new pull request, #65972: URL: https://github.com/apache/airflow/pull/65972
## Summary `breeze registry backfill` is currently broken on bare CI runners, every job in the `Registry Backfill` workflow fails before extracting any data. The backfill command's first step invokes: ``` uv run python dev/registry/extract_versions.py --provider X --version Y ``` with `cwd=AIRFLOW_ROOT_PATH` and no `--project` flag. uv discovers the workspace's `pyproject.toml` and tries to sync the entire workspace's deps, which transitively pulls in `samba -> smbprotocol -> pyspnego[kerberos] -> gssapi==1.11.1`. The gssapi build needs `krb5-config` (libkrb5-dev), which isn't installed on the GitHub `ubuntu-24.04` runner image, so the build fails: ``` × Failed to build `gssapi==1.11.1` ╰─▶ subprocess.CalledProcessError: Command 'krb5-config --libs gssapi' ``` The script reports `WARNING: extract_versions.py failed`, continues with `extract_parameters.py` + `extract_connections.py` (which work fine inside the breeze CI container), but the recorded failure flips the whole job to exit 1. The "Sync backfilled version pages to S3" step never runs, so nothing lands. ## Design rationale **Why `--project dev/registry` rather than `--no-project`?** `extract_versions.py` imports `yaml` and (transitively, via `registry_contract_models`) `pydantic` — both declared in `dev/registry/pyproject.toml`. `--no-project` would silently rely on whatever happens to be on the ambient PATH; `--project dev/registry` deterministically resolves the same deps the workflow YAML already uses. This brings the breeze invocation into line with the same fix applied to the `registry-build.yml` and `registry-backfill.yml` workflows in #63766. The workflow YAMLs were patched there but the equivalent invocation inside `breeze registry backfill` was missed. **Why the helper extraction?** The inline command moved into a private `_run_extract_versions(provider_id, version)` helper so the regression has a unit test. Four tests are added; one asserts `--project` is present and points to `DEV_REGISTRY_DIR`, so a future refactor can't silently re-introduce the workspace-discovery path. ## Gotchas / tradeoffs - The [`Extract version metadata from git tags`](https://github.com/apache/airflow/blob/main/.github/workflows/registry-backfill.yml#L179-L189) step in `registry-backfill.yml` already runs the same script correctly in a step *before* `breeze registry backfill`. Step 1 of the breeze command therefore re-runs the same extraction. That redundancy is left in place by this PR; removing it is a follow-up. The duplicate run is now fast (uv resolves from `dev/registry` deps, no workspace sync). - No fallback retry path (unlike `_run_extract_script`): this script's deps live in `dev/registry/pyproject.toml` and aren't provider-version dependent, so a single invocation either works or doesn't. ## Known issues / follow-up - The most-recent [`Registry Backfill` run](https://github.com/apache/airflow/actions/runs/24974797364) (2026-04-27) shows 90 of 90 backfill jobs failing identically with this error. After this PR merges, dispatchers can re-run the workflow to fill the historical version pages the previous attempt tried and failed to write. - The duplicate-extract noted above is a separate cleanup; tracking as future work, not in scope here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
