potiuk opened a new pull request, #71165:
URL: https://github.com/apache/airflow/pull/71165
The `all` extra of the `apache-airflow` meta-package was generated from
every provider in the source tree that is not suspended, so it also named the
not-ready ones. A not-ready provider has never been published, which makes the
extra impossible to satisfy against PyPI: there is no version of
`apache-airflow-providers-ibm-mq` to resolve to.
That is what broke constraints generation for 3.3.1rc1:
```
Because there are no versions of apache-airflow-providers-ibm-mq{...} and
apache-airflow[all]==3.3.0 depends on
apache-airflow-providers-ibm-mq{...}>=0.1.0,
we can conclude that apache-airflow[all]==3.3.0 cannot be used.
```
`ibm.mq` and `common.dataquality` are both `state: not-ready`; `apache.beam`
is `suspended` and was already excluded.
Two things had to change, because the second is what the first would
otherwise have papered over:
- `get_all_provider_ids()` grows an `exclude_not_ready_providers` flag, and
the pyproject generator passes it, so a not-ready provider is no longer listed
as a distribution in `all` and no longer gets an extra of its own.
- The self-referencing entry inside `all` —
`apache-airflow[async,pandas,...]` — was filtered against `all_providers`, i.e.
against the *included* providers. Excluding a provider from that list therefore
moved its extra into the self-reference rather than dropping it, and `all`
would still have pulled the provider in through the side door. It is now
filtered against every provider id whatever its state.
Non-provider extras are unaffected: `async`, `cloudpickle`, `graphviz`,
`kerberos`, `ldap`, `pandas`, `polars`, `sentry`, `statsd` and the rest are all
still there, as is `apache-airflow-core[all]`. Regenerating twice produces no
further change.
### Behaviour change worth noting
`[dependency-groups] dev` includes `apache-airflow[all]`, so the two
not-ready providers lose their `extra == 'all'` edge in `uv.lock` — that is
what the lock diff is. They are `editable = "providers/..."` entries resolved
from the workspace, so this was never a resolution problem locally; it only
bites when `[all]` is resolved against PyPI.
CI and breeze are unaffected: providers are installed with `uv sync
--all-packages` (`install_airflow_when_building_images.sh`,
`entrypoint_ci.sh`), which installs every workspace member regardless of state.
Source-providers constraints are unaffected too — they freeze the installed
environment and never reference `[all]`.
It also fixes `dev/get_devel_deps.sh`, which installs
`apache-airflow[all]==<version>` from PyPI and would hit the same unsatisfiable
resolution.
##### Was generative AI tooling used to co-author this PR?
- [X] Yes — Claude Opus 5 (1M context)
Generated-by: Claude Opus 5 (1M context) following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
