One question: Are you talking about removing things from
airflow.contrib, or things already with in airflow.providers.*?
-a
On Thu, Aug 4 2022 at 11:55:52 +02:00:00, Jarek Potiuk
<[email protected]> wrote:
Hello everyone,
Following the discussion in
<https://github.com/apache/airflow/pull/25413> I have a proposal.
Why don't we remove all "contrib" and other Airflow 1.10 deprecated
classes to a separate package and add dependency to that package as
[contrib] or [deprecated] extra in Airflow - and release one of the
next 2.* (2.4/2.5) airflow versions without those classes ?
This should be possible I think (We could do it for all "contrib"
easily and for some other individual classes in "operators" and
"hooks" that would likely require a little dynamic python magic to
not override the folders).
There are a number of benefits:
0) lots of old, defunc mostly deprecation code can be removed from
the main repo - including lots of tests that verify that.
1) new users would not even know about those classes/contrib - less
confusion
2) many of those "contrib" classes are not backwards-compatible with
old 1.10 classes already as we had many "major" releases in many
providers - so this might be a little misleading for those who are
still in 1.10 that they can easily "migrate" without making any
changes
3) there are many users who even now use "contrib" and we could use
the opportunity to "guide them" into migration. To make it smoother,
we could likely implement dynamic attribute checking in packages and
raise appropriate instructions to those who still use it and migrate
to 2.5 or 2.6 (and they will still have the option to install the
"contrib" package). The instructions could be the same as in
deprecation messages today (but they would fail in case the "contrib"
package is not installed)
4) We give a great tool for admins of Airflow installation. Currently
the admins have no tools to force their users to switch-off from
using contrib if they still do. But with this one they will simply be
able to install airflow without the "contrib" package and the users
will have to adapt. We can even provide those "Admins" instructions
on how to build your own "contrib" package if you want to do it
gradually and ask your users to remove class-by-class or whatever way
you want.
Technically - we are not breaking SemVer compatibility - you can
still get back to the contrib if you install the separate package. So
we can do it without bumping Major version
WDYT?
J.
