Hi Matthias, I'm not aware of any decision that has already been made by the community regarding after which 1.x minor release will we ship the 2.0 major release. I personally don't think it's feasible to avoid 1.19, and even 1.20 depending on the progress.
I also don't think we should push all the deprecation works in 1.18. In the "Deprecate multiple APIs in 1.18" thread [1], I only listed APIs that giving the impression already deprecated but are actually not fully (either in code or in documentation), in order to clarify the status and to properly deprecate the onse should be. We should not decide when to deprecate an existing API based on whether we would have a 1.19 or 1.20 minor release. Deciding to deprecate / remove an API definitely deserves thorough discussions and FLIP, which takes time, and I don't think we should compromise that for any reason. I think the potential conflict is between not being able to deprecate APIs very soon (needs more discussions, the new replacing API is not ready, etc.) and the wish to ship 2.0 on time. Assuming at some point we want to ship the 2.0 release, and there are some deprecated APIs that we want to remove but have not fulfilled the migration period required by FLIP-321 [2], I see 3 options to move forward. 1. Not removing the deprecated APIs in 2.0, carrying them until 3.0. This might be suitable if there are not a lot of deprecated APIs that we need to carry and the maintenance overhead is acceptable. 2. Postpone the 2.0 release for another minor release. This might be suitable if there are a lot of deprecated APIs. 3. Considering such APIs as exceptions of FLIP-321. This might be suitable if there are only a few of such APIs but have significant maintenance overhead. As discussed in FLIP-321, the introduction of the migration period effectively means we need to plan for the deprecation / removal of APIs early. As it is newly introduced and we haven't given developers the chance to plan things ahead, it's probably fair to make exceptions for API removal in the 2.0 version bump. My options are slightly different from what Chesnay has proposed. But I do agree that none of these options are great. I guess that's the price for not having the deprecation process earlier. Trying to deprecate things early is still helpful, because it reduces the number of APIs we need to consider when deciding between the options. However, that must not come at the price of rush decisions. I'd suggest developers to design / discuss / vote the breaking changes at their pace, and we evaluate the status and choose between the options later (maybe around the time releasing 1.19). If there are some contributors who think it makes sense, I will raise it in > the 1.18 release channel to postpone 1.18 feature freeze again. > I don't think postponing 1.18 would help a lot in this case, unless we postponed it for another couple of months. I don't think all the API changing plans can be finalized in a couple of weeks. Best, Xintong [1] https://lists.apache.org/thread/3dw4f8frlg8hzlv324ql7n2755bzs9hy [2] https://lists.apache.org/thread/vmhzv8fcw2b33pqxp43486owrxbkd5x9 On Thu, Jul 13, 2023 at 9:41 PM Jing Ge <j...@ververica.com.invalid> wrote: > Hi, > > Thanks Matthias for starting this discussion and thanks Chesnay for the > clarification. > > I don't want to hijack this discussion but I would suggest postponing the > 1.18 feature freeze over postponing the deprecations to 1.19. If there are > some contributors who think it makes sense, I will raise it in the 1.18 > release channel to postpone 1.18 feature freeze again. > > Speaking of the FLIP rules, if we follow it exactly like it is defined, we > should also write FLIPs when graduating @PublicEvloving APIs to be @Public, > especially for those APIs who will replace some deprecated APIs. Doing > that is to guarantee that new public APIs will cover all > functionalities(including the capability that APIs are easy enough to > implement) that deprecated APIs offer, so that migrations can be executed > smoothly. With this in mind, we will avoid the big issue we are facing now > wrt the new Source and Sink API [1]. > > Best regards, > Jing > > [1] https://lists.apache.org/thread/734zhkvs59w2o4d1rsnozr1bfqlr6rgm > > On Thu, Jul 13, 2023 at 3:01 PM Chesnay Schepler <ches...@apache.org> > wrote: > > > The issue isn't avoiding 1.19. > > > > The issue is that if things aren't deprecated in 1.18 then for every > > breaking change we have to start discussing exemptions from the API > > deprecation process, which stipulates that all Public APIs must be > > deprecated for at least 2 minor releases before they can be removed > > (which is now unsurprisingly backfiring on us). > > > > So if something isn't deprecated in 1.18 then either: > > - we delay 2.0 by at 1 release cycle > > - we effectively ignore the newly agreed upon deprecation process for 2.0 > > - we circumvent the newly agreed upon deprecation process by creating 2 > > minor releases in the same time-frame that we'd usually create 1 release > > in. > > > > None of these options are great. > > > > On 13/07/2023 14:03, Matthias Pohl wrote: > > > Jing brought up a question in the FLIP-335 discussion thread [1] which > I > > > want to move into a dedicated discussion thread as it's a bit more > > general: > > > How do we handle the deprecation process of Public APIs for Flink 2.0? > > >> I just have a related question: Do we need to create a FLIP each time > > >> when we want to deprecate any classes? > > > > > > The community defined the requirements of a FLIP [2] in the following > > way: > > > - Any major new feature, subsystem, or piece of functionality > > > - Any change that impacts the public interfaces of the project > > > > > > public interfaces in this sense are defined as: > > > - DataStream and DataSet API, including classes related to that, such > as > > > StreamExecutionEnvironment > > > - Classes marked with the @Public annotation > > > - On-disk binary formats, such as checkpoints/savepoints > > > - User-facing scripts/command-line tools, i.e. bin/flink, Yarn scripts, > > > Mesos scripts > > > - Configuration settings > > > - Exposed monitoring information > > > > > > I think that this makes sense. There should be a proper discussion on > the > > > best approach to change public APIs. Additionally, the FLIP should be a > > way > > > to document the changes in the discussion process towards the change > > > transparently. > > > > > > In contrast, the impression I have is that we're trying to push all the > > > deprecation work into 1.18 (which has its feature freeze date by the > end > > of > > > next week) to avoid having an additional 1.19 minor release. (Correct > me > > if > > > I'm wrong here but that's the impression I'm getting from the ML > threads > > > I've seen so far.) > > > > > > I have some concerns on the Flink 2.0 development in this regard. Zhu > Zhu > > > [4] and Chesnay [5] shared similar concerns in the thread about the 2.0 > > > must-have work items. > > > > > > Considering that quite a few (or some; I haven't checked in detail to > be > > > honest) of the changes for 2.0 should require a FLIP and that there are > > > still some hanging items [6] (Jira issues which are annotated for Flink > > 2.0 > > > but have been properly checked, yet): Shouldn't we avoid pushing > > everything > > > into 1.18? Instead, we should follow the required process properly and > > > might plan for another 1.19 minor release, instead? > > > > > > I'm curious how other contributors feel here and sorry in case I have > > > misinterpreted the ongoing discussions. > > > > > > Best, > > > Matthias > > > > > > [1] https://lists.apache.org/thread/48ysrg1rrtl8s1twg9wmx35l201hnc2w > > > [2] > > > > > > https://cwiki.apache.org/confluence/display/Flink/Flink+Improvement+Proposals#FlinkImprovementProposals-Whatisconsidereda%22majorchange%22thatneedsaFLIP > > > ? > > > [3] https://lists.apache.org/thread/r0y9syc6k5nmcxvnd0hj33htdpdj9k6m > > > [4] https://lists.apache.org/thread/45xm348jr8n6s89jldntv5z3t13hdbn8 > > > [5] https://lists.apache.org/thread/98wgqrx0sycpskvgpydvkywsoxt0fkc6 > > > [6] https://lists.apache.org/thread/77hj39ls3kxvx2qd6o09hq1ndtn6hg4y > > > > > > > >
