> based on the premise that your had to install the `dask` extra in the first place to get dask module of the right version
After reviewing the Dask Executor <https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/executor/dask.html> section in Airflow documentation, I didn't find any requirement or recommendation to add the `dask` extra when using it. So, including the dependencies explicitly in the requirements file should suffice to use the executor. This means that this change might potentially break existing setups for some users when they upgrade Airflow version, which contradicts the Airflow deprecation policy. If the premise holds true for Dask, it should also apply to Celery and Kubernetes. Nevertheless, we decided to pre-install their providers after the migration. However, this executor isn't as popular as the others, and as an Airflow user, I prefer not to have Dask dependencies in my environment when I install Airflow with the Celery executor. This helps avoid conflicts and enables me to upgrade libraries independently from their constraints. One possible solution could be allowing users to instruct Airflow not to pre-install executors providers by passing an argument via `--install-option` during pip install, and add install them explicitly by providing the extras. Since the flexibility of our deprecation policy isn't entirely clear to me, I'm neutral on this matter, with a vote of +0. On Fri, Jul 21, 2023 at 9:10 AM Elad Kalif <[email protected]> wrote: > I agree with Ash > -1 as well. > > > On Fri, Jul 21, 2023 at 9:29 AM Ash Berlin-Taylor <[email protected]> wrote: > > > -1 - based on the premise that your had to install the `dask` extra in > the > > first place to get dask module of the right version, so if we make the > > existing extra depends on the new provider then it's good enough. > > > > On 21 July 2023 06:22:28 BST, Jarek Potiuk <[email protected]> wrote: > > >Q: Do we want to pre-install Dask provider in Airflow 2.7.0 (with Dask > > >executor) once separated ? > > > > > >Discussion was here: > > >https://lists.apache.org/thread/0d9x4kl7hc2qzvho2mbdf35ohn5w12l6 > > > > > >Please vote: > > > > > >* +1 -> yes, we want to have dask provider preinstalled > > >* -1 -> no, it's fine to make it optional > > >* 0 -> no opinion > > > > > >Consider it my -1: I think we should NOT preinstall Dask provider. > > > > > >Voting guidelines here. This is really a "procedural" matter rather > > >than code modification: > > > > > >> Votes on procedural issues follow the common format of majority rule > > unless otherwise stated. That is, if there are more favourable votes than > > unfavourable ones, the issue is considered to have passed -- regardless > of > > the number of votes in each category. (If the number of votes seems too > > small to be representative of a community consensus, the issue is > typically > > not pursued. > > > > > >In this case committers have binding votes but other community members > > >are encouraged to state their non-binding votes as well. > > > > > >https://www.apache.org/foundation/voting.html > > > > > >--------------------------------------------------------------------- > > >To unsubscribe, e-mail: [email protected] > > >For additional commands, e-mail: [email protected] > > > > > >
