Hey, Zarzycki Thanks for your reply and recognization !
> As I understand you're thinking with Flink CDC 3.0 to make the project > broader than CDC, as more "Streaming Integration" framework. This will > resemble Kafka Connect, what about renaming the project to something like > "Flink Connect"? :-) > It would be amazing to have such a tool, "swiss army knife" for data > integration, based on Flink. Flink Community has a lot of connectors thus the Flink Connect name may not suitable. Although we offers end-to-end streaming integration capability , we still want to use the current name Flink CDC as we want to do best in database CDC area and users has accepted/known the name well. While the project name holds some significance, I believe its true importance lies in its ability to effectively address problems and deliver substantial value to the community users. Best, Leonard > > > On Tue, Dec 12, 2023 at 3:01 PM Lincoln Lee <lincoln.8...@gmail.com> wrote: > >> +1 >> Thanks Leonard and all CDC contributors for the great work! >> >> Best, >> Lincoln Lee >> >> >> Dian Fu <dian0511...@gmail.com> 于2023年12月12日周二 19:11写道： >> >>> Big +1 for this! Thanks for driving this effort! >>> >>> Regards, >>> Dian >>> >>> On Mon, Dec 11, 2023 at 4:48 PM Leonard Xu <xbjt...@gmail.com> wrote: >>>> >>>> Hey all, >>>> >>>> I want to extend my gratitude for the positive feedback we've received, >>> particularly from voices in other communities. It is an honor for the >> Flink >>> CDC community to have received your recognition. >>>> >>>> Moving forward, I will collaborate with the Flink CDC community >>> maintainers to prepare a FLIP (Flink Improvement Proposal) document that >>> will outline specific plans and implementation details. >>>> >>>> Thank you for your support, and we look forward to contributing to >>> Apache Flink community. >>>> >>>> Best, >>>> Leonard >>>> >>>>> 2023年12月11日 下午4:12，yh z <zhengyunhon...@gmail.com> 写道： >>>>> >>>>> Excited to hear the news, >>>>> +1 >>>>> >>>>> Best, >>>>> Yunhong (Swuferhong) >>>>> >>>>> liu ron <ron9....@gmail.com> 于2023年12月11日周一 14:05写道： >>>>> >>>>>> +1 >>>>>> >>>>>> Best, >>>>>> Ron >>>>>> >>>>>> Yunqing Mo <yunq...@apache.org> 于2023年12月11日周一 12:01写道： >>>>>> >>>>>>> So cool, Big +1 for this exciting work. >>>>>>> >>>>>>> On 2023/12/07 03:24:59 Leonard Xu wrote: >>>>>>>> Dear Flink devs, >>>>>>>> >>>>>>>> As you may have heard, we at Alibaba (Ververica) are planning to >>> donate >>>>>>> CDC Connectors for the Apache Flink project[1] to the Apache Flink >>>>>>> community. >>>>>>>> >>>>>>>> CDC Connectors for Apache Flink comprise a collection of source >>>>>>> connectors designed specifically for Apache Flink. These >>> connectors[2] >>>>>>> enable the ingestion of changes from various databases using Change >>> Data >>>>>>> Capture (CDC), most of these CDC connectors are powered by >>> Debezium[3]. >>>>>>> They support both the DataStream API and the Table/SQL API, >>> facilitating >>>>>>> the reading of database snapshots and continuous reading of >>> transaction >>>>>>> logs with exactly-once processing, even in the event of failures. >>>>>>>> >>>>>>>> >>>>>>>> Additionally, in the latest version 3.0, we have introduced many >>>>>>> long-awaited features. Starting from CDC version 3.0, we've built a >>>>>>> Streaming ELT Framework available for streaming data integration. >>> This >>>>>>> framework allows users to write their data synchronization logic >> in a >>>>>>> simple YAML file, which will automatically be translated into a >> Flink >>>>>>> DataStreaming job. It emphasizes optimizing the task submission >>> process >>>>>> and >>>>>>> offers advanced functionalities such as whole database >>> synchronization, >>>>>>> merging sharded tables, and schema evolution[4]. >>>>>>>> >>>>>>>> >>>>>>>> I believe this initiative is a perfect match for both sides. For >> the >>>>>>> Flink community, it presents an opportunity to enhance Flink's >>>>>> competitive >>>>>>> advantage in streaming data integration, promoting the healthy >>> growth and >>>>>>> prosperity of the Apache Flink ecosystem. For the CDC Connectors >>> project, >>>>>>> becoming a sub-project of Apache Flink means being part of a >> neutral >>>>>>> open-source community, which can attract a more diverse pool of >>>>>>> contributors. >>>>>>>> >>>>>>>> Please note that the aforementioned points represent only some of >>> our >>>>>>> motivations and vision for this donation. Specific future >> operations >>> need >>>>>>> to be further discussed in this thread. For example, the >> sub-project >>> name >>>>>>> after the donation; we hope to name it Flink-CDC aiming to >> streaming >>> data >>>>>>> intergration through Apache Flink, following the naming convention >> of >>>>>>> Flink-ML; And this project is managed by a total of 8 maintainers, >>>>>>> including 3 Flink PMC members and 1 Flink Committer. The remaining >> 4 >>>>>>> maintainers are also highly active contributors to the Flink >>> community, >>>>>>> donating this project to the Flink community implies that their >>>>>> permissions >>>>>>> might be reduced. Therefore, we may need to bring up this topic for >>>>>> further >>>>>>> discussion within the Flink PMC. Additionally, we need to discuss >>> how to >>>>>>> migrate existing users and documents. We have a user group of >> nearly >>>>>> 10,000 >>>>>>> people and a multi-version documentation site need to migrate. We >>> also >>>>>> need >>>>>>> to plan for the migration of CI/CD processes and other specifics. >>>>>>>> >>>>>>>> >>>>>>>> While there are many intricate details that require >> implementation, >>> we >>>>>>> are committed to progressing and finalizing this donation process. >>>>>>>> >>>>>>>> >>>>>>>> Despite being Flink’s most active ecological project (as evaluated >>> by >>>>>>> GitHub metrics), it also boasts a significant user base. However, I >>>>>> believe >>>>>>> it's essential to commence discussions on future operations only >>> after >>>>>> the >>>>>>> community reaches a consensus on whether they desire this donation. >>>>>>>> >>>>>>>> >>>>>>>> Really looking forward to hear what you think! >>>>>>>> >>>>>>>> >>>>>>>> Best, >>>>>>>> Leonard (on behalf of the Flink CDC Connectors project >> maintainers) >>>>>>>> >>>>>>>> [1] https://github.com/ververica/flink-cdc-connectors >>>>>>>> [2] >>>>>>> >>>>>> >>> >> https://ververica.github.io/flink-cdc-connectors/master/content/overview/cdc-connectors.html >>>>>>>> [3] https://debezium.io >>>>>>>> [4] >>>>>>> >>>>>> >>> >> https://ververica.github.io/flink-cdc-connectors/master/content/overview/cdc-pipeline.html >>>>>>> >>>>>> >>>> >>> >>
