> Not if you add install `airflow[contrib]` - then it won't break. Having to make a change to keep your system working throughout minor versions — to me that’s a breaking change.
So as much as we’d like to get rid of the contrib folder, we should really only change that throughout major versions. Bas > On 4 Aug 2022, at 12:46, Elad Kalif <elad...@apache.org> wrote: > > > Every time we release Amazon major version of provider, it might break if > you relied on it being backwards compatible. > > Why? The contrib just import the new file/class path. How can it be broken? > It has no functionality other than change the path > > On Thu, Aug 4, 2022 at 1:25 PM Jarek Potiuk <ja...@potiuk.com > <mailto:ja...@potiuk.com>> wrote: > And BTW. airflow/contrib/operators/ecs_operator.py is potentially already > broken multiple times - Every time we release Amazon major version of > provider, it might break if you relied on it being backwards compatible. > > On Thu, Aug 4, 2022 at 12:22 PM Jarek Potiuk <ja...@potiuk.com > <mailto:ja...@potiuk.com>> wrote: > Not if you add install `airflow[contrib]` - then it won't break. > > On Thu, Aug 4, 2022 at 12:21 PM Ash Berlin-Taylor <a...@apache.org > <mailto:a...@apache.org>> wrote: > Looking at the PR I see you are talking specifically about removing > airflow/contrib/operators/ecs_operator.py. > > I disagree with you that this doesn't count as a breaking change. > > For example, if we remove that file: > > I have a dag on Airflow 2.3. I upgrade to 2.4. My dag breaks. > > That is 100% a breaking change to me. > > -a > > On Thu, Aug 4 2022 at 11:08:14 +01:00:00, Ash Berlin-Taylor <a...@apache.org > <mailto:a...@apache.org>> wrote: >> One question: Are you talking about removing things from airflow.contrib, or >> things already with in airflow.providers.*? >> >> -a >> >> On Thu, Aug 4 2022 at 11:55:52 +02:00:00, Jarek Potiuk <ja...@potiuk.com >> <mailto:ja...@potiuk.com>> wrote: >>> Hello everyone, >>> >>> Following the discussion in https://github.com/apache/airflow/pull/25413 >>> <https://github.com/apache/airflow/pull/25413> I have a proposal. >>> >>> Why don't we remove all "contrib" and other Airflow 1.10 deprecated classes >>> to a separate package and add dependency to that package as [contrib] or >>> [deprecated] extra in Airflow - and release one of the next 2.* (2.4/2.5) >>> airflow versions without those classes ? >>> >>> This should be possible I think (We could do it for all "contrib" easily >>> and for some other individual classes in "operators" and "hooks" that would >>> likely require a little dynamic python magic to not override the folders). >>> >>> There are a number of benefits: >>> >>> 0) lots of old, defunc mostly deprecation code can be removed from the main >>> repo - including lots of tests that verify that. >>> >>> 1) new users would not even know about those classes/contrib - less >>> confusion >>> >>> 2) many of those "contrib" classes are not backwards-compatible with old >>> 1.10 classes already as we had many "major" releases in many providers - >>> so this might be a little misleading for those who are still in 1.10 that >>> they can easily "migrate" without making any changes >>> >>> 3) there are many users who even now use "contrib" and we could use the >>> opportunity to "guide them" into migration. To make it smoother, we could >>> likely implement dynamic attribute checking in packages and raise >>> appropriate instructions to those who still use it and migrate to 2.5 or >>> 2.6 (and they will still have the option to install the "contrib" package). >>> The instructions could be the same as in deprecation messages today (but >>> they would fail in case the "contrib" package is not installed) >>> >>> 4) We give a great tool for admins of Airflow installation. Currently the >>> admins have no tools to force their users to switch-off from using contrib >>> if they still do. But with this one they will simply be able to install >>> airflow without the "contrib" package and the users will have to adapt. We >>> can even provide those "Admins" instructions on how to build your own >>> "contrib" package if you want to do it gradually and ask your users to >>> remove class-by-class or whatever way you want. >>> >>> Technically - we are not breaking SemVer compatibility - you can still get >>> back to the contrib if you install the separate package. So we can do it >>> without bumping Major version >>> >>> WDYT? >>> >>> J. >>> >>> >>> >>> >>> >>>
