Re: [PROPOSAL] Standardize Airflow Packaging and build process (modern appropriate PEP-compliant tooling)

2024-01-10 Thread Jarek Potiuk
And merged :D. On Wed, Jan 10, 2024 at 6:56 PM Jarek Potiuk wrote: > I made a comparison of package files before <> after and added a few > corrections. Also I've added an extra security layer for CI building of > airflow packages - it runs inside a fully isolated Docker container. > > Would be

Re: [PROPOSAL] Standardize Airflow Packaging and build process (modern appropriate PEP-compliant tooling)

2024-01-10 Thread Jarek Potiuk
I made a comparison of package files before <> after and added a few corrections. Also I've added an extra security layer for CI building of airflow packages - it runs inside a fully isolated Docker container. Would be great to get another quick look /review before I merge it :) J, On Wed, Jan

Re: [PROPOSAL] Standardize Airflow Packaging and build process (modern appropriate PEP-compliant tooling)

2024-01-09 Thread Jarek Potiuk
Hey Everyone, I got the PR green: https://github.com/apache/airflow/pull/36537 - I got a really comprehensive review and a number of iterations with Jens (and approval! yay!!) and a number of comments from TP. I would love to have some feedback from others before merging, I still want to (I will

Re: [PROPOSAL] Standardize Airflow Packaging and build process (modern appropriate PEP-compliant tooling)

2024-01-07 Thread Jarek Potiuk
Ah .. .And comparing to the original proposal I simplified it a LOT. generally speaking for both contributor and user the way how you install Airflow for installation and contribution is "standard" and basically just "fixes" what has been broken - i.e. you just install it as expected: * `pip

Re: [PROPOSAL] Standardize Airflow Packaging and build process (modern appropriate PEP-compliant tooling)

2024-01-07 Thread Jarek Potiuk
Hello everyone, I iterated quite a bit on the PR and I think it's ready for an even more serious review: https://github.com/apache/airflow/pull/36537 . I solved all of the TODOs and teething problems and while it likely still has some tests to fix, all the build and packaging pieces, local

Re: [PROPOSAL] Standardize Airflow Packaging and build process (modern appropriate PEP-compliant tooling)

2024-01-04 Thread Jarek Potiuk
I slept over it a few nights and got away of it and I have an idea to simplify it quite a bit - i.e. cut the number of extras by half and virtually make 0 impact on current editable installation so you might wnnt to hold on a bit with that (unless you want to see it changing :) ) .. The whole

Re: [PROPOSAL] Standardize Airflow Packaging and build process (modern appropriate PEP-compliant tooling)

2024-01-04 Thread Pierre Jeambrun
I personally think that this is a great idea. I have been following the hatch project for a while and I am convinced it has a lot to offer for airflow. The two big pros for me are its ease of use (backend and front end) as well as the security covered aspects (reproducible builds to name one). I

[PROPOSAL] Standardize Airflow Packaging and build process (modern appropriate PEP-compliant tooling)

2024-01-02 Thread Jarek Potiuk
Hello everyone. Tl;DR; I have a proposal to adopt Hatchling as a build backend (and recommend, but not require Hatch as frontend) for Airflow as our way of switching to PEP-standard compliant pyproject.toml way of installing Airflow (including local venvs) and building the Airflow package. I