kaxil opened a new pull request, #65223:
URL: https://github.com/apache/airflow/pull/65223

   The registry backfill workflow had two problems preventing it from 
backfilling intermediate provider versions:
   
   1. **Flat version list applied to all providers**: Input like 
`providers="amazon google" versions="9.24.0 21.0.0"` tried all versions against 
all providers, failing because amazon doesn't have 21.0.0 and google doesn't 
have 9.24.0.
   
   2. **Missing system dependencies**: Extraction ran `uv run --with` on a bare 
GitHub runner, which lacks C libraries (krb5-dev, libxml2-dev) needed by 
providers like amazon and google.
   
   ## Changes
   
   **Workflow (`registry-backfill.yml`):**
   - Replace separate `providers` + `versions` inputs with `provider-versions` 
accepting `provider/version` pairs (e.g. `amazon/9.24.0 google/21.0.0 
amazon/9.23.0`)
   - jq-based matrix builder groups pairs by provider, so each job gets only 
its relevant versions
   - Add `build-ci-image` job and `prepare_breeze_and_image` step (same pattern 
as `registry-build.yml`)
   
   **Breeze command (`registry_commands.py`):**
   - Add `_backfill_docker()` that runs extraction inside the Breeze CI 
container via `pip install` + `execute_command_in_shell`
   - Refactor existing host-based extraction into `_backfill_uv()` as 
`--no-docker` fallback
   - Default is Docker; local dev can use `--no-docker` for faster iteration
   
   ## Example usage
   
   ```bash
   # Backfill multiple providers with different versions
   gh workflow run "Registry Backfill" \
     -f destination=live \
     -f provider-versions="amazon/9.24.0 google/21.0.0 celery/3.17.2"
   ```
   
   This creates parallel jobs: one for amazon (9.24.0), one for google 
(21.0.0), one for celery (3.17.2). Multiple versions per provider are grouped 
into a single job.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to