+1 for "API + binary compatibility for @PublicEvolving classes for all bug fix releases in a minor release (x.y.z is compatible to x.y.u)"
This @PublicEnvolving would then be a hard limit to changes. So it's important to rethink the policy towards using it, as Stephan proposed. I think any Flink interfaces that are visible to users should be explicitly marked as @Public or @PublicEnvolving. Any other interfaces should not be marked as @Public/@PublicEnvolving. This would be essential for us to check whether we are breaking any user faced interfaces unexpectedly. The only exception would be the case that we had to expose a method/class due to implementation limitations, it should be explicitly marked it as @Internal. Thanks, Zhu Zhu Yun Tang <myas...@live.com> 于2020年5月15日周五 上午11:41写道： > +1 for this idea, and I also like Xintong's suggestion to make it > explicitly when the @PublicEvolving API could upgrade to @Public API. > If we have the rule to upgrade API stable level but not define the clear > timeline, I'm afraid not everyone have the enthusiasm to upgrade this. > > The minor suggestion is that I think two major release (which is x.y.0 as > Chesnay clarified) might be a bit quick. From the release history [1], > Flink bump major version every 3 ~ 6 months and two major release gap > could only be at least half a year. > I think half a year might be a bit too frequent for users to collect > enough feedbacks, and upgrading API stable level every 3 major versions > should be better. > > [1] https://flink.apache.org/downloads.html#flink > > Best > Yun Tang > > > ________________________________ > From: Xintong Song <tonysong...@gmail.com> > Sent: Friday, May 15, 2020 11:04 > To: dev <dev@flink.apache.org> > Subject: Re: [DISCUSS] Stability guarantees for @PublicEvolving classes > > ### Documentation on API compatibility policies > > Do we have any formal documentation about the API compatibility policies? > The only things I found are: > > - In the release announcement (take 1.10.0 as an example) [1]: > "This version is API-compatible with previous 1.x releases for APIs > annotated with the @Public annotation." > - JavaDoc for Public [2] and PublicEvolving [3]. > > I think we might have a formal documentation, clearly state our policies > for API compatibility. > > - What does the annotations mean > - In what circumstance would the APIs remain compatible / become > incompatible > - How do APIs retire (e.g., first deprecated then removed?) > > Maybe there is already such kind of documentation that I overlooked? If so, > we probably want to make it more explicit and easy-to-find. > > ### @Public vs. @PublicEvolving for new things > > I share Stephan's concern that, with @PublicEvolving used for every new > feature and rarely upgraded to @Public, we are practically making no > compatibility guarantee between minor versions (x.y.* / x.z.*). On the > other hand, I think in many circumstances we do need some time to collect > feedbacks for new features before we have enough confidence to make the > commitment that our APIs are stable. Therefore, it makes more sense to me > to first make new features @PublicEvolving and then upgrade to @Public in > the next one or two releases (unless there's a good reason to further > postpone it). > > I think the key point is how do we make sure the @PublicEvolving features > upgrade to @Public. Maybe we can add a parameter to indicate the expected > upgrading version. E.g., a new feature introduced in release 1.10.0 might > be annotated as @PublicEvolving("1.12.0"), indicating that it is expected > to be upgraded to @Public in release 1.12.0. We can check the annotations > against the version automatically, forcing to either upgrad the feature > to @Public or explicitly postpone it by modifying the annotation parameter > (if there's a good reason). > > Additionally, we can do the similar for deprecated features / APIs, > reminding us to remove things annotated as @Deprecated at certain time. > > Thank you~ > > Xintong Song > > > [1] https://flink.apache.org/news/2020/02/11/release-1.10.0.html > > [2] > > https://ci.apache.org/projects/flink/flink-docs-master/api/java/org/apache/flink/annotation/Public.html > > [3] > > https://ci.apache.org/projects/flink/flink-docs-master/api/java/org/apache/flink/annotation/PublicEvolving.html > > > > > On Thu, May 14, 2020 at 8:22 PM Stephan Ewen <se...@apache.org> wrote: > > > I just want to throw in that we also need to rethink our policy towards > > using @PublicEvolving. > > > > We often introduce this easily (for every new feature) and rarely (almost > > never) upgrade it to @Public. This kind of leads the idea behind stable > API > > guarantees ad absurdum. > > > > I would suggest that we make @PublicEvolving an exception that needs a > good > > reason rather than for everything that is new (when we don't want to be > > bothered with thinking about compatibility). > > > > > > On Thu, May 14, 2020 at 1:05 PM Xintong Song <tonysong...@gmail.com> > > wrote: > > > > > Thanks for the clarification. > > > +1 for keeping the current guarantees for @Public. > > > > > > Thank you~ > > > > > > Xintong Song > > > > > > > > > > > > On Thu, May 14, 2020 at 6:07 PM Till Rohrmann <trohrm...@apache.org> > > > wrote: > > > > > > > Sorry for the confusion. @Public classes are guaranteed to be stable > > > > between releases x.y.z and x.u.v (minor and bug fix release; naming > is > > > > indeed a bit off here) and we can break it with major releases (x.0.0 > > and > > > > y.0.0). > > > > > > > > @Tison I would like to make what to include in the public API, hence > > what > > > > to annotate with @Public and @PublicEvolving, a separate discussion. > > > > > > > > Cheers, > > > > Till > > > > > > > > On Thu, May 14, 2020 at 11:48 AM tison <wander4...@gmail.com> wrote: > > > > > > > >> Thanks for starting this discussion! > > > >> > > > >> I agree turn on japicmp on PublicEvolving among bugfix releases is a > > nit > > > >> win. > > > >> > > > >> @Xintong Song <tonysong...@gmail.com> I think @Public guarantee is > > good > > > >> enough, the problem is a reachable 2.0 plan. > > > >> > > > >> My concern is more on classes that have no annotation but our > > developers > > > >> regard as "something that should be stable". Previously I was > required > > > to > > > >> keep compatibility of ClusterClient & HighAvailabilityServices > because > > > >> they > > > >> might be depended on by user. > > > >> > > > >> Best, > > > >> tison. > > > >> > > > >> > > > >> Dawid Wysakowicz <dwysakow...@apache.org> 于2020年5月14日周四 下午5:08写道： > > > >> > > > >> > I also like the proposal for keeping the binary compatibility of > > > >> > @PublicEvolving for bugfix releases. > > > >> > > > > >> > As for the @Public classes I think the current guarantees are good > > > >> enough. > > > >> > > > > >> > Best, > > > >> > > > > >> > Dawid > > > >> > > > > >> > On 14/05/2020 10:49, Jingsong Li wrote: > > > >> > > Thanks Till for starting this discussion. > > > >> > > > > > >> > > +1 for enabling the japicmp-maven-plugin for @PublicEvolving for > > bug > > > >> fix > > > >> > > releases. > > > >> > > Bug fix should just be user imperceptible bug fix. Should not > > affect > > > >> API > > > >> > > and binary compatibility. > > > >> > > > > > >> > > And even PublicEvolving api change for "y" release, we should > > expose > > > >> it > > > >> > in > > > >> > > dev mail list for discussing or a FLIP? > > > >> > > > > > >> > > BTW, public api can be changed by major releases? In annotation > > > >> comments: > > > >> > > "Only major releases (1.0, 2.0, 3.0) can break interfaces with > > this > > > >> > > annotation". > > > >> > > > > > >> > > Best, > > > >> > > Jingsong Lee > > > >> > > > > > >> > > On Thu, May 14, 2020 at 4:30 PM Till Rohrmann < > > trohrm...@apache.org > > > > > > > >> > wrote: > > > >> > > > > > >> > >> Dear community, > > > >> > >> > > > >> > >> in the latest 1.10.1 bug fix release I introduced a binary > > > >> incompatible > > > >> > >> change to a class which is annotated with @PublicEvolving [1]. > > > While > > > >> > this > > > >> > >> change is technically ok since we only provide API and binary > > > >> > compatibility > > > >> > >> for @Public classes across releases, it raised the question > > whether > > > >> we > > > >> > >> can't do better. > > > >> > >> > > > >> > >> For our users it might be surprising and really annoying that > > they > > > >> > cannot > > > >> > >> simply upgrade to the latest bug fix release without > recompiling > > > the > > > >> > >> program or even having to change the source code of an > > > application. I > > > >> > >> believe we would provide a much better experience if we ensured > > > that > > > >> bug > > > >> > >> fix releases maintain API and binary compatibility also for > > > >> > @PublicEvolving > > > >> > >> classes. Hence my proposal would be to tighten the stability > > > >> guarantees > > > >> > the > > > >> > >> following way: > > > >> > >> > > > >> > >> * API + binary compatibility for @Public classes across all > > > releases > > > >> > (x.y.z > > > >> > >> is compatible to u.v.w) > > > >> > >> * API + binary compatibility for @PublicEvolving classes for > all > > > bug > > > >> fix > > > >> > >> releases in a minor release (x.y.z is compatible to x.y.u) > > > >> > >> > > > >> > >> This would entail that we can change @PublicEvolving classes > only > > > >> across > > > >> > >> minor/major releases. > > > >> > >> > > > >> > >> Practically this would mean that we enable the > > japicmp-maven-plugin > > > >> > >> for @PublicEvolving for bug fix releases. > > > >> > >> > > > >> > >> What do you think? > > > >> > >> > > > >> > >> [1] > > > >> > >> > > > >> > >> > > > >> > > > > >> > > > > > > https://lists.apache.org/thread.html/r293768d13d08149d756e0bf91be52372edb444c317535d1d5a496c3e%40%3Cdev.flink.apache.org%3E > > > >> > >> > > > >> > >> Cheers, > > > >> > >> Till > > > >> > >> > > > >> > > > > > >> > > > > >> > > > > >> > > > > > > > > > >
