potiuk opened a new issue, #54609: URL: https://github.com/apache/airflow/issues/54609
Currently constraints are used for three purposes: 1) to take snapshot of latest "working" version of dependencies 2) to limit PRs to "latest known good constraints" - so that newly released packages are not failing most PRs, but fail canary builds instead 3) to give our users a way to install released airflow in reproducible way However, after we switched to uv, the case 2) can be better achieve with uv.lock. Currently we .gitignore uv.lock, but we could commit it to the repository and handle constraint generation differently * constraints could be generated using latest lock * canary builds could use the --resolution highest as today and diverge from latest "frozen" uv.lock * all prs would use the uv.lock and do not use --resolution-highest in those PRs. This has the added benefit that `uv sync` in PR would automatically upgrade dependencies if the dependencies in pyproject.toml are conflicting with uv.lock - but it would not use `--resolution highest`, which means that the PRs that modify dependnecies do not have to get latest dependencies when they are run - that would limit the number of PRs that are experiencing "main/canary" problems The only problem to solve is who and when would upgrade the uv.lock and commit it to the repo. Currently constraints are automatically bumped after successful canary runs, but they are in orphan un-protected branches, while uv.lock shares the code with main airflow repo, and we need a "human with ICLA" to review and merge commits there (ASF requirement) Proposed solution is to add such uv.lock upgrade to our "upgrade-important-versions" prek hook. We run it in CI in canary builds and we fail the builds when new versions of important installers change (pip, Python base images, golang) - this happens often-enough to be eventually-consistent pretty fast, but rarely enough to make it into annoing "every day" upgrade chore. Benefits: * simplification of breeze build image with constraints (currently it uses complex `uv pip install` with --constraint, and it could simply use `uv sync --all-packages` instead if our uv.lock is checked in and maintained * better synchronization of the local venv. Currently `uv sync` does not guarantee to have the same version of dependencies in local venv. The only way to guarantee it now is to use `breeze` image - build with constraints or run complex command with constraints added when you run `uv sync` * less "3rd-party release" impact on failing PRs. if PRs will not touch the dependency specification where the new release is released, it will continue using the dependency stored in .lock and not the latest dependency released Cons: Probably we should pay better attention to run `upgrade-important-versions` regularly and even add an option there to manually just perfom sync (for example that will be useful as part of the release procedure, where release manager would have to make sure to run the `upgrade-important-versions` check when preparing release notes. But it has the advantage that it will be a conscious step that should actually help in making sure that constraints are updated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
