+1 for this exciting work
Best regards, WT | | bai_wentao1 | | bai_went...@163.com | ---- Replied Message ---- | From | ConradJam<jam.gz...@gmail.com> | | Date | 12/8/2023 14:26 | | To | <dev@flink.apache.org> | | Subject | Re: [PROPOSAL] Contribute Flink CDC Connectors project to Apache Flink | +1 Best Idea . Thanks to all the contributors of Flink CDC for the great work. yuxia <luoyu...@alumni.sjtu.edu.cn> 于2023年12月7日周四 20:46写道: +1 for this. Thanks all the contributors of Flink CDC for the great work. Best regards, Yuxia ----- 原始邮件 ----- 发件人: "Martijn Visser" <martijnvis...@apache.org> 收件人: "dev" <dev@flink.apache.org> 发送时间: 星期四, 2023年 12 月 07日 下午 8:14:36 主题: Re: [PROPOSAL] Contribute Flink CDC Connectors project to Apache Flink Hi Leonard, +1 for this. I think the CDC connectors are a great result and example of the Flink CDC community. Kudos to you all. Best regards, Martijn On Thu, Dec 7, 2023 at 10:40 AM Márton Balassi <balassi.mar...@gmail.com> wrote: Hi Leonard, Thank you for the excellent work you and the team working on the CDC connectors project have been doing so far. I am +1 of having them under Flink's umbrella. On Thu, Dec 7, 2023 at 10:26 AM Etienne Chauchot <echauc...@apache.org> wrote: Big +1, thanks this will be a very useful addition to Flink. Best Etienne Le 07/12/2023 à 09:26, Hang Ruan a écrit : +1 for contributing CDC Connectors to Apache Flink. Best, Hang Yuxin Tan<tanyuxinw...@gmail.com> 于2023年12月7日周四 16:05写道: Cool, +1 for contributing CDC Connectors to Apache Flink. Best, Yuxin Jing Ge<j...@ververica.com.invalid> 于2023年12月7日周四 15:43写道: Awesome! +1 Best regards, Jing On Thu, Dec 7, 2023 at 8:34 AM Sergey Nuyanzin<snuyan...@gmail.com wrote: thanks for working on this and driving it +1 On Thu, Dec 7, 2023 at 7:26 AM Feng Jin<jinfeng1...@gmail.com> wrote: This is incredibly exciting news, a big +1 for this. Thank you for the fantastic work on Flink CDC. We have created thousands of real-time integration jobs using Flink CDC connectors. Best, Feng On Thu, Dec 7, 2023 at 1:45 PM gongzhongqiang < gongzhongqi...@apache.org wrote: It's very exciting to hear the news. +1 for adding CDC Connectors to Apache Flink ! Best, Zhongqiang Leonard Xu<xbjt...@gmail.com> 于2023年12月7日周四 11:25写道: Dear Flink devs, As you may have heard, we at Alibaba (Ververica) are planning to donate CDC Connectors for the Apache Flink project *[1]* to the Apache Flink community. CDC Connectors for Apache Flink comprise a collection of source connectors designed specifically for Apache Flink. These connectors *[2]* enable the ingestion of changes from various databases using Change Data Capture (CDC), most of these CDC connectors are powered by Debezium *[3]* . They support both the DataStream API and the Table/SQL API, facilitating the reading of database snapshots and continuous reading of transaction logs with exactly-once processing, even in the event of failures. Additionally, in the latest version 3.0, we have introduced many long-awaited features. Starting from CDC version 3.0, we've built a Streaming ELT Framework available for streaming data integration. This framework allows users to write their data synchronization logic in a simple YAML file, which will automatically be translated into a Flink DataStreaming job. It emphasizes optimizing the task submission process and offers advanced functionalities such as whole database synchronization, merging sharded tables, and schema evolution *[4]*. I believe this initiative is a perfect match for both sides. For the Flink community, it presents an opportunity to enhance Flink's competitive advantage in streaming data integration, promoting the healthy growth and prosperity of the Apache Flink ecosystem. For the CDC Connectors project, becoming a sub-project of Apache Flink means being part of a neutral open-source community, which can attract a more diverse pool of contributors. Please note that the aforementioned points represent only some of our motivations and vision for this donation. Specific future operations need to be further discussed in this thread. For example, the sub-project name after the donation; we hope to name it Flink-CDC aiming to streaming data intergration through Apache Flink, following the naming convention of Flink-ML; And this project is managed by a total of 8 maintainers, including 3 Flink PMC members and 1 Flink Committer. The remaining 4 maintainers are also highly active contributors to the Flink community, donating this project to the Flink community implies that their permissions might be reduced. Therefore, we may need to bring up this topic for further discussion within the Flink PMC. Additionally, we need to discuss how to migrate existing users and documents. We have a user group of nearly 10,000 people and a multi-version documentation site need to migrate. We also need to plan for the migration of CI/CD processes and other specifics. While there are many intricate details that require implementation, we are committed to progressing and finalizing this donation process. Despite being Flink’s most active ecological project (as evaluated by GitHub metrics), it also boasts a significant user base. However, I believe it's essential to commence discussions on future operations only after the community reaches a consensus on whether they desire this donation. Really looking forward to hear what you think! Best, Leonard (on behalf of the Flink CDC Connectors project maintainers) [1]https://github.com/ververica/flink-cdc-connectors [2] https://ververica.github.io/flink-cdc-connectors/master/content/overview/cdc-connectors.html [3]https://debezium.io [4] https://ververica.github.io/flink-cdc-connectors/master/content/overview/cdc-pipeline.html -- Best regards, Sergey -- Best ConradJam