potiuk commented on PR #48226:
URL: https://github.com/apache/airflow/pull/48226#issuecomment-2750976221

   Well. This was not the reason in this case. The reason was really the new 
`setuptools` -  78.0.1 - started to complain about the invalid metadata. But 
even latest kylinpy 2.8.4 has this invalid metadata:
   
   ```cfg
   [aliases]
   test = pytest
   
   [metadata]
   description-file = README.rst    # !!!! THIS IS WRONG. Should be 
description_file !!!!
   
   [egg_info]
   tag_build = 
   tag_date = 0
   ```
   
   The problem with `setuptools` is that this is a default build-requirement 
for many packages. No matter what version you have locally, if your project has 
`pyproject.toml`, when a package needs to be built, it is built in 
`isolation_mode` by default - which means that whatever frontend you use 
(`pip`, `hatch`, `uv`) - it will create a separate, NEW virtualenv to build 
such project (`kylinpy` in this case) - it will install there LATEST 
requirements that the project declares as build dependencies and attempt to 
build the project in that venv.
   
   There are MANY projects like `kylinpy` that do not pin their build 
requirements, This means tha whenever new setuptools is released, unless you 
specifiy `--no-build-isolation` for specific package (there are ways to 
individually specify which packages should be build without isolation, they 
will have a separately installed venv with latest setuptools - and when the 
latest setuptools does not like bad metadata, such package cannot be built.
   
   There are several ways to deal with it:
   
   1) --no-build-isolation + pinning setuptools in your main venv (then such 
build will use whatever version of setuptools you already have in your local, 
shared venv where you run `pip install` or `uv pip install`
   
   2) when you use `pip` you can set PIP_CONSTRAINTS env variable and point it 
to a constraint file where you specify which version of setuptools (and other 
dependencies) should be used. Note - that only work via env variable, when you 
pass constraints with `--constraints` , those constrints are NOT passed to the 
isolated, new virtualenv
   
   3) when you use `uv` you can pass `--build-constraints` flag and pass 
constratints this way to the isolated venv where package is built. But then 
that flag applies to the whole `uv pip install` or `uv sync` or whatever 
command - so those constraints apply to all the packages that `uv` installs 
that needs building (i.e. all packages that have no .whl dstribution that 
matches your machine / architecture and so on.
   
   So in this case - when next time setuptools maintainers will attempt to get 
rid of old, deprecated and "wrong" metadata packges, we either have to 
workaround it in one of the ways above (but then our users will have to do the 
same when installing airflow from `PyPI` - which is not ideal. Or better - we 
need to work with those dependencies of ours that have that problem and either 
help to (F)ix them or (F)ork them (I did it with json-merge-patch where I 
vendored in the 70 lines of code) or (F)orego them - i.e. switch to another 
dependency that is modern.
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to