kaxil commented on PR #53149: URL: https://github.com/apache/airflow/pull/53149#issuecomment-3090229079
> I just started playing with it and the problem is that we miss `force-include` directive in our pyproject.toml. Hatchling has this built-in feature specifically foreseen for including files in the packages that are coming from "outside" of the project repo (which indeed is generally a security issue and unless you specifically set it in pyproject.toml should not happen). > > I tested it and adding this: > > ```toml > [tool.hatch.build.targets.sdist.force-include] > "../shared/timezones/src/airflow_timezones/timezone.py" = "src/airflow/sdk/_shared/timezone.py" > "../shared/logging/src/airflow_logging/logging" = "src/airflow/sdk/_shared/logging" > > [tool.hatch.build.targets.wheel.force-include] > "../shared/timezones/src/airflow_timezones/timezone.py" = "src/airflow/sdk/_shared/timezone.py" > "../shared/logging/src/airflow_logging/logging" = "src/airflow/sdk/_shared/logging" > ``` > > to task-sdk's pyproject.toml solves the problem - and includes both logging and shared in the final packages (both sdist and wheel will have files instead of symbolic links). And the same can be used for sharing code between shared modules. > > And it's a 'proper' solution that will work independently on how you are building the packages - does not matter if you use hatch or setuptools or `python -m build` this will work fine because this is handled by the "build backend" - i.e. in our pyproject.toml we specify that `hatchling` is our build backend, and any build frontend will use the version of hatchling we specified to buiold the packages. > > It really works in this way that first step of the build is that backend "isolated environment" is prepared, then backend copies all the sources to temporary directory (and does any kind of pre-processing - in this case it takes files specified by force-include and copies them to the right place in the temporary source tree) and then the packaging happens. So no matter which build frontend you use - it will **just work**. Cool, now the other thing that remains for parity is not being able to call another shared library from within an existing one. Example to be able to use `timezone` in "logging/structlog.py" file without using try..except etc -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
