Hi folks,

Merry Christmas 🎄🎅.

Currently, Airflow allows installing providers via "extras" -
https://airflow.apache.org/docs/apache-airflow/stable/extra-packages-ref.html#providers-extras
for
convenience.

We initially did this for easing Migrations for users from 1.10.x to 2.x.
But I want to discuss this again as I feel users can unintentionally
upgrade the providers versions.

Example:

pip install -U "apache-airflow[google]==2.2.3" -c $CONSTRAINTS_URL


Now, if a new major version of the provider is released between Airflow
2.2.2 and 2.2.3 and the user uses that command, it will bump the
"apache-airflow-providers-google" providers from as en example 4.0.0 to
5.0.0 and *all of its dependencies*.

This might have unintended consequences. Now an easy solution is to
downgrade the provider version to the previously installed version by
running if the user notices this:

pip install -U "apache-airflow-providers-google==4.0.0"


However, I feel we can stop this unintended upgrade by not allowing the
installation of providers via extras. This would also clear out any
confusion users might have on installing providers as we will only
have a *single
way* to install them and truly separate providers from the core. And users
can upgrade each provider only when they need to and asses when upgrading
to major versions of the provider.

On the flip side, installing providers via extras is actually really
convenient 😁 and I use them all the time for testing.

Thoughts?

Regards,
Kaxil

Reply via email to