Hi Alex, I saw that I missed replying to this topic. I do think that Xintong touched on an important topic when he mentioned that we should define what an LTS version means. From my point of view, I would state that an LTS version for Apache Flink means that bug fixes only will be made available for a longer period of time. I think that, combined with what you called option 1 (a clear end-of-life date) is the best option.
Flink 2.0 will give us primarily the ability to remove a lot of deprecated APIs, especially with Flink's deprecation strategy. I expect that the majority of users will have an easy migration path from a Flink 1.x to a Flink 2.0, if you're currently not using a deprecated API and are a Java user. Allowing backporting of new features to Flink 1.x will result in users delaying the upgrade, but it doesn't make the upgrade any easier when they must upgrade. New features will also introduce new bugs, meaning that maintainers will have to spend time on two release versions. As the codebases diverge more and more, this will just become increasingly more complex. With that being said, I do think that it makes sense to also formalize the result of this discussion in a FLIP. That's just easier to point users towards at a later stage. Best regards, Martijn On Mon, Dec 4, 2023 at 9:55 PM Alexander Fedulov <alexander.fedu...@gmail.com> wrote: > > Hi everyone, > > As we progress with the 1.19 release, which might potentially (although not > likely) be the last in the 1.x line, I'd like to revive our discussion on > the > LTS support matter. There is a general consensus that due to breaking API > changes in 2.0, extending bug fixes support by designating an LTS release > is > something we want to do. > > To summarize, the approaches we've considered are: > > Time-based: The last release of the 1.x line gets a clear end-of-life date > (2 years). > Release-based: The last release of the 1.x line gets support for 4 minor > releases in the 2.x line. The exact time is unknown, but we assume it to be > roughly 2 years. > LTS-to-LTS release: The last release of the 1.x line is supported until the > last release in the 2.x line is designated as LTS. > > We need to strike a balance between being user-friendly and nudging people > to > upgrade. From that perspective, option 1 is my favorite - we all know that > having a clear deadline works wonders in motivating action. At the same > time, > I appreciate that we might not want to introduce new kinds of procedures, > so > option 2 would be my second choice, provided we also formulate it like "4 > minor releases, at least 2 years" (in case the minor release cadence > accelerates for some reason). I find option 3 a bit problematic since it > gives no incentive to upgrade, and people who do not follow Flink > development > closely might be caught by surprise when we decide to bump the major > version > again. > > I'd like to open a vote to stamp the official decision, but first, I would > like to understand if we can reach consensus on one of the options here, or > if > we'll need to push that to a vote by presenting multiple options. > > Looking forward to hearing your thoughts on this matter. > > P.S.: 1.x and 2.x are just examples in this case; the decision also > translates > into a procedure for future major releases. > > Best, > Alex > > On Thu, 27 Jul 2023 at 17:32, Jing Ge <j...@ververica.com.invalid> wrote: > > > Hi Konstantin, > > > > What you said makes sense. Dropping modules is an efficient option to > > accelerate Flink evolution with acceptable function regressions. We should > > do it constantly and strategically. On the other hand, we should point out > > the core modules that must be backward compatible when a new major version > > is released. > > > > Best regards, > > Jing > > > > On Wed, Jul 26, 2023 at 10:52 PM Matthias Pohl > > <matthias.p...@aiven.io.invalid> wrote: > > > > > > > > > > @Mathias, I am not quite sure about the 3 versions description. Are you > > > > concerned that 1.x and 2.x LTS releases could overlap, if 3.0 comes > > > early? > > > > > > Yes. Maybe, that's only a theoretical scenario. It wouldn't work if we go > > > with your suggestion to use "proper time" rather than release cycles to > > > define the length of a support period (which sounds reasonable). My > > concern > > > was that we get into a situation where we need to support four versions > > of > > > Flink. > > > > > > On Wed, Jul 26, 2023 at 4:21 PM Alexander Fedulov < > > > alexander.fedu...@gmail.com> wrote: > > > > > > > The question is if we want to tie the release cycle of 2.x to how much > > > time > > > > we give our users to migrate. And "time" is a critical word here. I can > > > see > > > > us > > > > potentially wanting to iterate on the 2.x line more rapidly, because of > > > all > > > > of the > > > > major changes, until the cycles get settled to a typical cadence again. > > > > > > > > This means that user's won't know how much time they would have to > > > actually > > > > migrate off of 1.x. And I can see this knowledge being critical for > > > > companies' > > > > quarterly/yearly plannings, so transparency here is key. > > > > > > > > That's why I think it makes sense to deviate from the typical N minor > > > > releases > > > > rule and set an explicit time period. We usually have a minor release > > > every > > > > four > > > > months, so my proposition would be to designate a 1.5 years period as > > > > a generous approximation of a 4 releases cycle. > > > > > > > > I also agree with limiting support to bugfixes - Flink is at the level > > of > > > > maturity where > > > > I believe nothing so critical will be missing in the last 1.x release > > > that > > > > we'd need to > > > > backport if from 2.x. In the end, we want to encourage users to > > migrate. > > > > > > > > @Mathias, I am not quite sure about the 3 versions description. Are you > > > > concerned that 1.x and 2.x LTS releases could overlap, if 3.0 comes > > > early? > > > > > > > > Best, > > > > Alex > > > > > > > > On Wed, 26 Jul 2023 at 14:47, Matthias Pohl <matthias.p...@aiven.io > > > > .invalid> > > > > wrote: > > > > > > > > > I think making the last minor release before a major release an LTS > > > > release > > > > > with extended support makes sense. I cannot think of a reason against > > > the > > > > > four minor release cycles suggested by Marton. Only providing bug > > fixes > > > > and > > > > > not allowing features to be backported sounds reasonable to keep the > > > > > maintenance costs low. > > > > > > > > > > And maybe we can make this a general convention for last minor > > releases > > > > for > > > > > > all major releases, rather than only discuss it for the 2.0 version > > > > bump. > > > > > > > > > > > > > > > I guess we should also make it explicit that we only support one LTS > > > > > version to reduce the number of supported versions to 3 (x.y, > > x.[y-1], > > > > > [x-1].z). I have the impression that this is implied by everyone. I > > > > wanted > > > > > to mention this as an additional item, anyway. ...even though it > > would > > > > only > > > > > become a topic when discussing 3.0. > > > > > > > > > > Matthias > > > > > > > > > > On Tue, Jul 25, 2023 at 6:33 PM Jing Ge <j...@ververica.com.invalid> > > > > > wrote: > > > > > > > > > > > Hi Konstantin, > > > > > > > > > > > > I might have not made myself clear enough, apologies. The > > > > > > source-/sink-function was used as a concrete example to discuss the > > > > > pattern > > > > > > before we decided to offer LTS. The intention was not to hijack > > this > > > > > thread > > > > > > to discuss how to deprecate them. > > > > > > > > > > > > We all wish that the only thing users need to migrate from Flink > > 1.x > > > to > > > > > 2.0 > > > > > > is some code changes in their repos and we all wish users will > > > migrate, > > > > > if > > > > > > LTS has long enough support time. But the question I tried to > > discuss > > > > is > > > > > > not the wish but the "How?". We might be able to toss the high > > > > migration > > > > > > effort aside(we shouldn't), since it is theoretically still doable > > if > > > > > users > > > > > > have long enough time, even if the effort is extremely high. > > Another > > > > > > concern is that if "function regressions" is allowed in 2.0, i.e. > > if > > > > 2.0 > > > > > > has a lack of functionalities or bugs compared to 1.x, there will > > be > > > no > > > > > way > > > > > > for users to do the migration regardless of whether we encourage > > them > > > > to > > > > > > migrate or they haven been given enough time(how long is enough?) > > > > because > > > > > > LTS has been offered. How could we help users and avoid this > > > happening? > > > > > > > > > > > > Best regards, > > > > > > Jing > > > > > > > > > > > > On Tue, Jul 25, 2023 at 6:57 PM Konstantin Knauf < > > kna...@apache.org> > > > > > > wrote: > > > > > > > > > > > > > Hi Jing, > > > > > > > > > > > > > > let's not overindex on the Source-/SinkFunction discussion in > > this > > > > > > thread. > > > > > > > > > > > > > > We will generally drop/break a lot of APIs in Flink 2.0. So, > > > > naturally > > > > > > > users will need to make more changes to their code in order to > > > > migrate > > > > > > from > > > > > > > 1.x to Flink 2.0. In order to give them more time to do this, we > > > > > support > > > > > > > the last Flink 1.x release for a longer time with bug fix > > releases. > > > > > > > > > > > > > > Of course, we still encourage users to migrate to Flink 2.0, > > > because > > > > at > > > > > > > some point, we will stop support Flink 1.x. For example, if we > > > > followed > > > > > > > Marton's proposal we would support Flink 1.x LTS for about 2 > > years > > > > > > (roughly > > > > > > > 4 minor release cycles) instead of about 1 year (2 minor release > > > > > cycles) > > > > > > > for regular minor releases. This seems like a reasonable > > timeframe > > > to > > > > > me. > > > > > > > It also gives us more time to discover and address blockers in > > > > > migrating > > > > > > to > > > > > > > Flink 2.x that we are not aware of right now. > > > > > > > > > > > > > > Best, > > > > > > > > > > > > > > Konstantin > > > > > > > > > > > > > > Am Di., 25. Juli 2023 um 12:48 Uhr schrieb Jing Ge > > > > > > > <j...@ververica.com.invalid>: > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > Overall, it is a good idea to provide the LTS release, but I'd > > > like > > > > > to > > > > > > > > reference a concrete case as an example to understand what > > > > > restrictions > > > > > > > the > > > > > > > > LTS should have. > > > > > > > > > > > > > > > > Hypothetically, Source-/Sink- Function have been deprecated in > > > 1.x > > > > > LTS > > > > > > > and > > > > > > > > removed in 2.0 and the issues[1] are not solved in 2.0. This > > is a > > > > > > typical > > > > > > > > scenario that the old APIs are widely used in 1.x LTS and the > > new > > > > > APIs > > > > > > in > > > > > > > > 2.0 are not ready yet to take over all users. We will have the > > > > > > following > > > > > > > > questions: > > > > > > > > > > > > > > > > 1. Is this scenario allowed at all? Do we all agree that there > > > > could > > > > > be > > > > > > > > some features/functionalities that only work in 1.x LTS after > > 2.0 > > > > has > > > > > > > been > > > > > > > > released? > > > > > > > > 2. How long are we going to support 1.x LTS? 1 year? 2 years? > > As > > > > long > > > > > > as > > > > > > > > the issues that block users from migrating to 2.0 are not > > solved, > > > > we > > > > > > > can't > > > > > > > > stop the LTS support, even if the predefined support time > > > expires. > > > > > > > > 3. What is the intention to release a new version with (or > > > without) > > > > > > LTS? > > > > > > > Do > > > > > > > > we still want to engage users to migrate to the new release > > asap? > > > > If > > > > > > the > > > > > > > > old APIs 1.x LTS offer more than the new APIs in 2.0 or it is > > > > almost > > > > > > > > impossible to migrate, double effort will be required to > > maintain > > > > > those > > > > > > > > major releases for a very long time. We will be facing many > > > > cohorts. > > > > > > > > > > > > > > > > IMHO, we should be clear with those questions before we start > > > > talking > > > > > > > about > > > > > > > > LTS. WDYT? > > > > > > > > > > > > > > > > Best regards, > > > > > > > > Jing > > > > > > > > > > > > > > > > > > > > > > > > [1] > > > > https://lists.apache.org/thread/734zhkvs59w2o4d1rsnozr1bfqlr6rgm > > > > > > > > > > > > > > > > On Tue, Jul 25, 2023 at 6:08 PM Márton Balassi < > > > > > > balassi.mar...@gmail.com > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Hi team, > > > > > > > > > > > > > > > > > > +1 for supporting the last 1.x for a longer than usual period > > > of > > > > > time > > > > > > > and > > > > > > > > > limiting it to bugfixes. I would suggest supporting it for > > > double > > > > > the > > > > > > > > usual > > > > > > > > > amount of time (4 minor releases). > > > > > > > > > > > > > > > > > > On Tue, Jul 25, 2023 at 9:25 AM Konstantin Knauf < > > > > > kna...@apache.org> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Hi Alex, > > > > > > > > > > > > > > > > > > > > yes, I think, it makes sense to support the last 1.x > > release > > > > > longer > > > > > > > > than > > > > > > > > > > usual. This should be limited to bugfixes in my opinion. > > > > > > > > > > > > > > > > > > > > Best, > > > > > > > > > > > > > > > > > > > > Konstantin > > > > > > > > > > > > > > > > > > > > Am Di., 25. Juli 2023 um 07:07 Uhr schrieb Xintong Song < > > > > > > > > > > tonysong...@gmail.com>: > > > > > > > > > > > > > > > > > > > > > Hi Alex, > > > > > > > > > > > > > > > > > > > > > > Providing a longer supporting period for the last 1.x > > minor > > > > > > release > > > > > > > > > makes > > > > > > > > > > > sense to me. > > > > > > > > > > > > > > > > > > > > > > I think we need to be more specific about what LTS means > > > > here. > > > > > > > > > > > > > > > > > > > > > > - IIUC, that means for the last 1.x minor release, we > > > will > > > > > > keep > > > > > > > > > > > providing 1.x.y / 1.x.z bugfix release. This is a > > > stronger > > > > > > > support > > > > > > > > > > > compared > > > > > > > > > > > to regular minor releases which by default are only > > > > > supported > > > > > > > for > > > > > > > > 2 > > > > > > > > > > > minor > > > > > > > > > > > release cycles. > > > > > > > > > > > - Do we only provide bug fixes for the LTS release, or > > > do > > > > we > > > > > > > also > > > > > > > > > > allow > > > > > > > > > > > backporting features to that release? > > > > > > > > > > > - How long exactly shall we support the LTS release? > > > > > > > > > > > > > > > > > > > > > > And maybe we can make this a general convention for last > > > > minor > > > > > > > > releases > > > > > > > > > > for > > > > > > > > > > > all major releases, rather than only discuss it for the > > 2.0 > > > > > > version > > > > > > > > > bump. > > > > > > > > > > > > > > > > > > > > > > @Leonard, > > > > > > > > > > > > > > > > > > > > > > I'd like to clarify that there are no community decisions > > > yet > > > > > on > > > > > > > > > release > > > > > > > > > > > 2.0 after 1.19. It is possible to have 1.20 before 2.0. > > > > > > > > > > > > > > > > > > > > > > Best, > > > > > > > > > > > > > > > > > > > > > > Xintong > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Jul 25, 2023 at 11:54 AM Leonard Xu < > > > > xbjt...@gmail.com > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > +1, it’s pretty necessary especially we deprecated so > > > many > > > > > APIs > > > > > > > in > > > > > > > > > 1.18 > > > > > > > > > > > > and plan to remove in 2.0. > > > > > > > > > > > > > > > > > > > > > > > > The 1.19 should be a proper version for LTS Release. > > > > > > > > > > > > > > > > > > > > > > > > Best, > > > > > > > > > > > > Leonard > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Jul 25, 2023, at 3:30 AM, Alexander Fedulov < > > > > > > > > > > > > alexander.fedu...@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > Hello everyone, > > > > > > > > > > > > > > > > > > > > > > > > > > Recently, there were a lot of discussions about the > > > > > > deprecation > > > > > > > > of > > > > > > > > > > > > various > > > > > > > > > > > > > APIs for the upcoming 2.0 release. It appears there > > are > > > > two > > > > > > > main > > > > > > > > > > > > motivations > > > > > > > > > > > > > with opposing directions, causing these discussions > > to > > > > > remain > > > > > > > > > > > unsettled. > > > > > > > > > > > > On > > > > > > > > > > > > > one hand, there's a desire to finally trim a wide > > range > > > > of > > > > > > > legacy > > > > > > > > > > APIs, > > > > > > > > > > > > some > > > > > > > > > > > > > lingering around since the beginning of the 1.x > > release > > > > > line > > > > > > > (as > > > > > > > > > far > > > > > > > > > > > > back as > > > > > > > > > > > > > 2016). On the other hand, there is a commitment to > > > uphold > > > > > our > > > > > > > > > > > guarantees > > > > > > > > > > > > to > > > > > > > > > > > > > the users, ensuring a smooth transition. > > > > > > > > > > > > > > > > > > > > > > > > > > I believe we could reconcile these two motivations. > > My > > > > > > > > proposition > > > > > > > > > is > > > > > > > > > > > to > > > > > > > > > > > > > designate the final release of the 1.x timeline as a > > > > > > Long-Term > > > > > > > > > > Support > > > > > > > > > > > > (LTS) > > > > > > > > > > > > > release. By doing so, we would: > > > > > > > > > > > > > > > > > > > > > > > > > > 1. Enable more efficient cleanup and be liberated to > > > > > > introduce > > > > > > > > more > > > > > > > > > > > > breaking > > > > > > > > > > > > > changes, paving the way for greater innovation in > > the > > > > 2.0 > > > > > > > > > release. > > > > > > > > > > > > > 2. Sustain a positive user experience by granting > > > enough > > > > > time > > > > > > > for > > > > > > > > > the > > > > > > > > > > > > > changes > > > > > > > > > > > > > introduced in 2.0 to stabilize, allowing users to > > > > > > confidently > > > > > > > > > > > > transition > > > > > > > > > > > > > their production code to the new release. > > > > > > > > > > > > > > > > > > > > > > > > > > I look forward to hearing your thoughts on this > > > proposal. > > > > > > > > > > > > > > > > > > > > > > > > > > Best Regards, > > > > > > > > > > > > > Alex > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > https://twitter.com/snntrable > > > > > > > > > > https://github.com/knaufk > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > https://twitter.com/snntrable > > > > > > > https://github.com/knaufk > > > > > > > > > > > > > > > > > > > > > > > > > > >
