Thanks for starting the discussion. I think both issues are valid concerns that we need to tackle. I guess the biggest issue is that now it's just not possible to write 1 connector that runs for Flink 1.13 and 1.14, so we make it much harder for devs in the ecosystem (and our goal is to make it easier!).
I think the only solution is to actually treat (and mark) all these interfaces as Public and only change them in Flink 2.0. If we need to change them, we would need to actually add a secondary (possibly internal) interface (at least that would have helped in SourceReaderContext). On Tue, Sep 28, 2021 at 3:20 PM Chesnay Schepler <ches...@apache.org> wrote: > We already have such tooling via japicmp; it's just that it is only > enabled for Public APIs. > > You can probably generate a report via japicmp for all > PublicEvolging/Experimental APIs as well. > > On 28/09/2021 15:17, Ingo Bürk wrote: > > Hi everyone, > > > > I think it would be best to support this process with tooling as much as > > possible, because humans are bound to make mistakes. FLINK-24138[1] > should > > be a first step in this direction, but it wouldn't catch the cases > > discussed here. > > Maybe we should consider "capturing" the public API into some separate > > file(s) and validating against that in the CI such that structural > changes > > to public APIs (moved classes, renamed / exchanged classes, changed > > signatures, …) can be caught as an error. That would raise awareness for > > any changes to public APIs and force a conscious decision. > > > > [1] https://issues.apache.org/jira/browse/FLINK-24138 > > > > > > Best > > Ingo > > > > On Tue, Sep 28, 2021 at 2:49 PM Leonard Xu <xbjt...@gmail.com> wrote: > > > >>>> Not sure if this will happen in 1.15 already. We will needed automated > >>>> compatibility tests and a well-defined list of stable API. > >>> We are > >>> trying to provide forward compatibility: applications using `@Public` > >> APIs > >>> compiled against Flink 1.12.x, should work fine in Flink 1.13.x > >> Unfortunately, I also meet forward compatibility issue, when I do the > >> release 1.14 check, I try to use mysql-cdc connector[1] which compiled > >> against 1.13.1in SQL Client, but it can not work in flink 1.14.0 > cluster, > >> it failed due to the metric API compatibility broken. > >> > >> @Public > >> public interface SourceReaderContext { > >> > >> MetricGroup metricGroup(); > >> > >> > >> @Public > >> public interface SourceReaderContext { > >> > >> SourceReaderMetricGroup metricGroup(); > >> > >> > >> Shouldn't we mark it as @Deprecated and then delete it util 2.0.0 > version > >> for @Public API as the our community rule [2] described? At least we > should > >> keep them across server minor versions (<major>.<minor>.<patch>). > >> > >> Although these changes can be tracked to voted FLIPs and it’s not the > >> fault of a few developers, it show us the fact that we didn’t pay enough > >> attention to back compatibility/forward compatibility. > >> > >> Best, > >> Leonard > >> [1] > >> > https://github.com/ververica/flink-cdc-connectors/tree/master/flink-connector-mysql-cdc > >> [2] > >> https://cwiki.apache.org/confluence/display/FLINK/Stability+Annotations > >> > >> > >
