Hi Xintong, 2. SourceFunction / SinkFunction This is one with challenges. There were many discussions in the past.
2.1 SourceFunction The voting was passed[1], but there are still some ongoing issues[2]. The SourceFunction is not marked as @deprecated yet[3], afaik. 2.2 SinkFunction SinkV2 is a complex one and e.g. even the most important implementation KafkaSink has not graduated to @Public yet[4]. FlieSink is @Experimental and there were some concerns with data loss issues when I proposed to graduate it to @PublicEvolving. It deserves a more serious graduation process. Otherwise, users would not migrate to SinkV2 and all SinkFunction and related implementations could not be removed. You can find more details and other issues from Dong's and my replies at [5] Best regards, Jing [1] https://lists.apache.org/thread/hrpsddgz65hjvhjozhg72s0wsmxz145p [2] https://github.com/apache/flink/pull/20049 [3] https://github.com/apache/flink/blob/c11fd82e0cc63904b43d2aca3a79d85dfeb57c2f/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/source/SourceFunction.java#L98 [4] https://github.com/apache/flink-connector-kafka/blob/910fc5a3f06ab7d95f79c0fc4b5ce64e2aaa8db2/flink-connector-kafka/src/main/java/org/apache/flink/connector/kafka/sink/KafkaSink.java#L57 [5] https://lists.apache.org/thread/q62nj89rrz0t5xtggy5n65on95f2rmmx On Thu, Jun 29, 2023 at 10:45 AM Martijn Visser <martijnvis...@apache.org> wrote: > Hi Xintong, > > With regards to the deprecation of the Scala APIs, during the PR > review it was requested to not mark all APIs as deprecated but only > the entry point [1], to avoid a fully striked through experience in > the IDE. I think the same idea was applicable on the DataSet API. I > think it depends on how formal we want to treat this: if really > formal, then we should deprecate them in 1.18. I think in both cases, > it's quite well known that they are deprecated. I'm +0 for either way, > as long as we're all agreeing that they can be removed in 2.0. > > With regards to Queryable State and Source/SinkFunction, +1 to mark > these as deprecated. > > Best regards, > > Martijn > > [1] > https://github.com/apache/flink/pull/21176#pullrequestreview-1159706808 > > On Thu, Jun 29, 2023 at 10:23 AM Xintong Song <tonysong...@gmail.com> > wrote: > > > > Hi devs, > > > > Looking at the release 2.0 proposals [1], I noticed that many APIs that > are > > proposed to be removed in 2.0 are not (fully) deprecated yet. We might > want > > to properly mark them as `@Deprecated` in 1.18 if we agree they should be > > removed in 2.0. Moreover, according to FLIP-321 [2] (not voted yet but > IMO > > is close to consensus IMO), a migration period is required after APIs are > > deprecated and before they can be removed. > > > > I might not be familiar with the status of all the APIs below. So I'd > like > > to bring them up and see if there's any concern regarding deprecating > them > > in 1.18. If there's concern for deprecating API, we can start a separate > > discussion thread for it. For those with no objections, I'd create JIRA > > tickets and try to properly deprecate them in 1.18. > > > > 1. DataSet API > > It's described as "legacy", "soft deprecated" in user documentation [3]. > > However, it's not annotated with `@Deprecated` in codes. According to > > FLIP-131 [4], DataSet API should be deprecated when DataStream API and > > Table API / SQL meet certain requirements. AFAICS, all the requirements > > mentioned in the FLIP are already fulfilled. We should annotate it as > > `@Deprecated` now. > > > > 2. SourceFunction / SinkFunction > > They are described as deprecated in the roadmap[5], and I don't find > > anything regarding them in user documentation. But they are also not > > annotated with `@Deprecated` in codes. TBH, I'm not aware of any formal > > decision to deprecate these. AFAICS, the replacement for SourceFunction > > (Source) has already been promoted to `@Public`, while the replacement > for > > SinkFunction (SinkV2) is still `@PublicEvolving`. I found a discussion[6] > > regarding promoting SinkV2 to `@Public`, but it's unclear to me what the > > conclusion is. > > > > 3. Queryable State > > It's described as approaching end-of-life in the roadmap [5], but is > > neither deprecated in codes nor in user documentation [7]. I also found a > > discussion [8] about rescuing it from deprecation, and it seems to me > there > > are more negative opinions than positive ones. > > > > 4. All Scala APIs > > I think we agreed to drop Scala API support in FLIP-265 [9], and have > tried > > to deprecate them in FLINK-29740 [10]. Also, both user documentation and > > roadmap[5] shows that scala API supports are deprecated. However, AFAICS, > > none of the APIs in `flink-streaming-scala` are annotated with > > `@Deprecated`, and only `ExecutionEnvironment` and `package` are marked > > `@Deprecated` in `flink-scala`. > > > > Looking forward to your feedback. > > > > Best, > > > > Xintong > > > > > > [1] https://cwiki.apache.org/confluence/display/FLINK/2.0+Release > > > > [2] https://lists.apache.org/thread/vmhzv8fcw2b33pqxp43486owrxbkd5x9 > > > > [3] > > > https://nightlies.apache.org/flink/flink-docs-master/docs/dev/dataset/overview/ > > > > [4] > > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=158866741 > > > > [5] https://flink.apache.org/roadmap/ > > > > [6] https://lists.apache.org/thread/q62nj89rrz0t5xtggy5n65on95f2rmmx > > > > [7] > > > https://nightlies.apache.org/flink/flink-docs-master/docs/dev/datastream/fault-tolerance/queryable_state/ > > > > [8] https://lists.apache.org/thread/9hmwcjb3q5c24pk3qshjvybfqk62v17m > > > > [9] > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-265+Deprecate+and+remove+Scala+API+support > > > > [10] https://issues.apache.org/jira/browse/FLINK-29740 >
