"separate POM" -> Separate pyproject.toml :D,, I've been in too many discussions in Apache Software Foundation about java dependencies and POM files and Maven (ough!) it seems :D
On Thu, Oct 17, 2024 at 12:22 PM Jarek Potiuk <ja...@potiuk.com> wrote: > Hello here, > > As discussed last week on slack, I prepared a comprehensive document > describing the current state of extras in Airflow (or at least my view on > it :) ). > > > https://docs.google.com/document/d/1e1T3u34bqo7yYSJVvq1NlEJJhA5_hUNXOXo7J6KFg_0/edit?usp=sharing > > The document describes the current state of extras as we use in Airflow 2 > (and same usage so far in Airflow 3) - split into development and user use > cases (which roughly splits to "standard" and "editable" installation. It > describes the different behaviour of extras in "standard" and "editable" > environments (via hatch_build.py hooks) as well as potential future and > very recent developments that might impact the way we will think about > extras for Airflow 3. > > For example the recent development about - accepted last week PEP 735 that > might change the way we will define development groups. But tools need to > catch up. > > Also the move to "uv workspaces" might (and will) change some of the ways > we interact with Airflow - especially for development, which might render > some of the extra use cases obsolete, but IMHO we need to do some > preparatory work (and further splitting providers to per-provider > sub-directories and each provider having a separate POM is a prerequisite > for that). > > There are also some other potential changes and future work we can do (all > marked in yellow) that might impact the way we think about and define our > extras. > > Finally - at the end - it contains a summary of all the use-cases (I found > 6 in total) how extras are currently used - user and development use cases. > > I know that there are voices that maybe we should get rid of some of the > extras - so having summary of the use cases might be helpful to discuss it: > > * whether the use case is actually "useful" > * whether we have some way to replace the use of extras to achieve similar > behaviour > > I propose - if you have detailed comments to the document - let's discuss > it there (everyone has commenter access by following the link). If you have > some proposals on what to do after reading and discussing details - we > should discuss any proposals in this thread > > Have a nice read :) > > J. > > >
