Any comments from others :) ? This is not something urgent, I think the most important part here is to "catch" the moment where maintaining providers to keep backwards compatibility becomes a burden. So far I Thiink there are no "huge" problems. Some common utils and some name changes. but I do not see anything "serious" enough to break the limits
But maybe there are some strong points. J, On Sat, Jan 8, 2022 at 12:10 PM Jarek Potiuk <ja...@potiuk.com> wrote: > Thanks Elad, > > Important point indeed. > > Agree this might be frustrating, but on the other hand we have to make > sure Airflow is maintainable in the long term - this means for example > that in a year or two we will have Airflow 2.0, Airflow 2.1, Airflow > 2.2, Airflow 2.3, Airflow 2.4. Airflow 2.5 lines. Does it mean that we > will be releasing fixes to all the 'minor' lines ? We are already in > the situation that we have not released a single fix to 2.0 once 2.1 > is released, we have not released a single fix to 2.1 once 2.2 was > released and we've never released a single fix to 2.2 once 2.3 was > released. Does it mean that 2.0, 2.1 and 2.2 are bug-free? Nope. A lot > of bugs that were also present in 2.0 were released in 2.3. > > As I see our current "de-facto" policy is - whenever you need > something new OR you see a bug in 2.0, 2.1, 2.2 - unless there is a > workaround - migrate to the latest 2.3. So we already encourage our > users to go with the latest version (which is good). > > So for example releasing providers that are 2.3+ once we have 2.4 or > 2.5 seems to me like a natural extension of that approach. And yeah - > one of the important points of providers is that you can upgrade them > independently. And it should stay like that. I would never propose a > "fast" increase of min version. I think however that at some point > getting such a "bump" in min-version makes sense. > > How often ? When? I am not sure, but I am pretty sure it should happen > from time to time. > > However I do agree it might surprise some users that some providers > will not get more fixes for the version of airflow they have, and > surprise is never a good thing. > > Just a though maybe - what could help is just agreeing on the rules > now and communicating them would be a good thing to do? > > Same as we agreed on Python version/K8S version policies. If we do > some kind of agreement (for example - we support providers for - say 1 > year after the first minor release of Airflow all providers we > guarantee that all providers released will be compatible with that > line? > > If we agree that 1 year is enough that would meaan: > > * 31 May 2022 - we bump min version to 2.2 (1 year after 2.1.0 was > released) > * 11 October 2022 - we bump min version to 2.3 (1 year after 2.2.0 was > released) > > We can change it to 1.5 year or even 2 years if we think 1 year is > enough, but I think 1 year should be pretty good. > > That would give a really nice cadence, it would be very > straightforward rule, easy to understand and reason about, it will > also give as an opportunity to periodically modernize our providers - > on the other hand it would give the users a significant period where > they can expect regular provider's release and enough time to prepare > for bigger migration. > > Also this is somewhat a community-imposed upgrade cadence. Which I > think is fair, reasonable and the only way we can actually (as a > community) give our users certainty that they will get a good level of > support from the community. It's really a trade - " You get the > software from us, we agree that we will maintain it with fixes, but in > return, you should follow this upgrade cadence if you want to get the > full support". This is just "fair". On the other hand I think it's > quite "unfair" from the users to expect from the community an infinite > time of support. Making clear rules on what to expect and rules that > will allow the community to provide a good level of support to the > users seems like a good idea. > > WDYT? > > J. > > On Wed, Jan 5, 2022 at 5:39 PM Elad Kalif <elad...@apache.org> wrote: > > > > We need to be also mindful about the fact that not everyone is able to > migrate to the new Airflow version as soon as it's being released. > > One of the concepts of providers is that it's not tightly bound to > Airflow core - If we can give users more time before setting a minimum > version of airflow 2.3 that would be ideal. > > It's frustrating when a small bug fix that you really need is beyond > your reach due to some other issue that forced bumping version. It may be > more hussle for us but if we can release provider bug fixes/features first > and a few days later release the breaking changes in a separate release it > may be valuable for some users. > > > > On Thu, Dec 30, 2021 at 8:11 PM Kaxil Naik <kaxiln...@gmail.com> wrote: > >> > >> One other solution for just using "utils" from core is we can release > "airflow.backports" or "airflow.futures" package same as Python Backports > and Python Future where we can handle the compat shim without restricting > Airflow versions. > >> > >> Regards, > >> Kaxil > >> > >> On Thu, Dec 30, 2021 at 2:56 PM Jarek Potiuk <ja...@potiuk.com> wrote: > >>> > >>> > At this point, I think 2.3.0 will be around early Feb to get AIP-42 > in. We can reassess how the progress is moving ahead with the AIP mid-Jan > and based on that we can either release 2.3.0 end of Jan without AIP-42 > changes and mark it for 2.4.0. > >>> > >>> Yeah Agree here. I do not think we need to make a decision now - we > >>> can do it later. But I'd love to hear various opinions - from our > >>> users, various stakeholders (Amazon/Google/Astronomer who run it as a > >>> service for one). > >>> > >>> Just to be clear - I am not fully convinced myself if this is the > >>> right time to do it already (but there are some good indicators that > >>> we should at least start discussing it). > >>> > >>> This is more opening the discussion and gathering feedback/feelings of > >>> people. I think such decisions should be discussed (and their > >>> consequences realized) well in-advance so that at the time of decision > >>> we have all arguments and opinions and know the consequences. > >>> > >>> And it is not "strictly" necessary. It's just (like with all similar > >>> cases) an increased burden for maintenance and extra/boilerplate code > >>> that might be removed. In those cases there is rarely a "0/1" decision > >>> that can be made "here we know for sure that we should increase > >>> min-version" - it's more of the collective thinking: "Are the > >>> burden/overhead heavy enough we don't want to carry it any more as a > >>> community". > >>> > >>> J. >
