Hi Becket, Thanks for driving this important topic! There were many discussions previously that ended up with waiting up for a clear API deprecation process definition. This FLIP will help a lot.
I'd like to ask some questions to understand your thoughts. Speaking of the FLIP, *"Always add a "Since X.X.X" comment to indicate when was a class / interface / method marked as deprecated."* Could you describe it with a code example? Do you mean Java comments? *"At least 1 patch release for the affected minor release for Experimental APIs"* The rule is absolutely right. However, afaiac, deprecation is different as modification. As a user/dev, I would appreciate, if I do not need to do any migration work for any deprecated API between patch releases upgrade. BTW, if experimental APIs are allowed to change between patches, could we just change them instead of marking them as deprecated and create new ones to replace them? One major issue we have, afaiu, is caused by the lack of housekeeping/house cleaning, there are many APIs that were marked as deprecated a few years ago and still don't get removed. Some APIs should be easy to remove and others will need some more clear rules, like the issue discussed at [1]. Some common questions could be: 1. How to make sure the new APIs cover all functionality, i.e. backward compatible, before removing the deprecated APIs? Since the functionalities could only be built with the new APIs iteratively, there will be a while (might be longer than the migration period) that the new APIs are not backward compatible with the deprecated ones. 2. Is it allowed to remove the deprecated APIs after the defined migration period expires while the new APis are still not backward compatible? 3. For the case of core API upgrade with downstream implementations, e.g. connectors, What is the feasible deprecation strategy? Option1 bottom-up: make sure the downstream implementation is backward compatible before removing the deprecated core APIs. Option2 top-down: once the downstream implementation of new APIs works fine, we can remove the deprecated core APIs after the migration period expires. The implementation of the deprecated APIs will not get any further update in upcoming releases(it has been removed). There might be some missing features in the downstream implementation of new APIs compared to the old implementation. Both options have their own pros and cons. Best regards, Jing [1] https://lists.apache.org/thread/m3o48c2d8j9g5t9s89hqs6qvr924s71o On Mon, Jun 12, 2023 at 6:31 PM Stefan Richter <srich...@confluent.io.invalid> wrote: > Hi, > > Thanks a lot for bringing up this topic and for the initial proposal. As > more and more people are looking into running Flink as a continuous service > this discussion is becoming very relevant. > > What I would like to see is a clearer definition for what we understand by > stability and compatibility. Our current policy only talks about being able > to “compile” and “run” with a different version. As far as I can see, there > is no guarantee about the stability of observable behavior. I believe it’s > important for the community to include this important aspect in the > guarantees that we give as our policy. > > For all changes that we do to the stable parts of the API, we should also > consider how easy or difficult different types of changes would be for > running Flink as a service with continuous delivery. For example, > introducing a new interface to evolve the methods would make it easier to > write adapter code than changing method signatures in-place on the existing > interface. Those concerns should be considered in our process for evolving > interfaces. > > Best, > Stefan > > > > <https://www.confluent.io/> > Stefan Richter > Principal Engineer II > > Follow us: < > https://www.confluent.io/blog?utm_source=footer&utm_medium=email&utm_campaign=ch.email-signature_type.community_content.blog> > <https://twitter.com/ConfluentInc> > > > > > On 11. Jun 2023, at 14:30, Becket Qin <becket....@gmail.com> wrote: > > > > Hi folks, > > > > As one of the release 2.0 efforts, the release managers were discussing > our > > API lifecycle policies. There have been FLIP-196[1] and FLIP-197[2] that > > are relevant to this topic. These two FLIPs defined the stability > guarantee > > of the programming APIs with various different stability annotations, and > > the promotion process. A recap of the conclusion is following: > > > > Stability: > > @Internal API: can change between major/minor/patch releases. > > @Experimental API: can change between major/minor/patch releases. > > @PublicEvolving API: can change between major/minor releases. > > @Public API: can only change between major releases. > > > > Promotion: > > An @Experimental API should be promoted to @PublicEvolving after two > > releases, and a @PublicEvolving API should be promoted to @Public API > after > > two releases, unless there is a documented reason not to do so. > > > > One thing not mentioned in these two FLIPs is the API deprecation > process, > > which is in fact critical and fundamental to how the stability guarantee > is > > provided in practice, because the stability is all about removing > existing > > APIs. For example, if we want to change a method "ResultFoo > foo(ArgumentFoo > > arg)" to "ResultBar bar(ArgumentBar arg)", there will be two ways to do > > this: > > > > 1. Mark method "foo" as deprecated and add the new method "bar". At some > > point later, remove the method "foo". > > 2. Simply change the API in place, that basically means removing method > foo > > and adding method bar at the same time. > > > > In the first option, users are given a period with stability guarantee to > > migrate from "foo" to "bar". For the second option, this migration period > > is effectively 0. A zero migration period is problematic because end > users > > may need a feature/bug fix from a new version, but cannot upgrade right > > away due to some backwards compatible changes, even though these changes > > perfectly comply with the API stability guarantees defined above. So the > > migration period is critical to the API stability guarantees for the end > > users. > > > > The migration period is essentially how long a deprecated API can be > > removed from the source code. So with this FLIP, I'd like to kick off the > > discussion about our deprecation process. > > > > > https://www.google.com/url?q=https://cwiki.apache.org/confluence/display/FLINK/FLIP-321%253A%2BIntroduce%2Ban%2BAPI%2Bdeprecation%2Bprocess&source=gmail-imap&ust=1687091523000000&usg=AOvVaw0h_j72PiGBNM3BFmuHUOis > > > > Comments are welcome! > > > > Thanks, > > > > Jiangjie (Becket) Qin > > > > [1] > > > https://www.google.com/url?q=https://cwiki.apache.org/confluence/display/FLINK/FLIP-196%253A%2BSource%2BAPI%2Bstability%2Bguarantees&source=gmail-imap&ust=1687091523000000&usg=AOvVaw0VDOsdIOFCOsGLlpPJq-ZD > > [2] > > > https://www.google.com/url?q=https://cwiki.apache.org/confluence/display/FLINK/FLIP-197%253A%2BAPI%2Bstability%2Bgraduation%2Bprocess&source=gmail-imap&ust=1687091523000000&usg=AOvVaw11GfC5R4cD44a8amORO8EY > >
