+1 for contributing CDC Connectors to Apache Flink. Best, Hang
Yuxin Tan <tanyuxinw...@gmail.com> 于2023年12月7日周四 16:05写道： > Cool, +1 for contributing CDC Connectors to Apache Flink. > > Best, > Yuxin > > > Jing Ge <j...@ververica.com.invalid> 于2023年12月7日周四 15:43写道： > > > Awesome! +1 > > > > Best regards, > > Jing > > > > On Thu, Dec 7, 2023 at 8:34 AM Sergey Nuyanzin <snuyan...@gmail.com> > > wrote: > > > > > thanks for working on this and driving it > > > > > > +1 > > > > > > On Thu, Dec 7, 2023 at 7:26 AM Feng Jin <jinfeng1...@gmail.com> wrote: > > > > > > > This is incredibly exciting news, a big +1 for this. > > > > > > > > Thank you for the fantastic work on Flink CDC. We have created > > thousands > > > of > > > > real-time integration jobs using Flink CDC connectors. > > > > > > > > > > > > Best, > > > > Feng > > > > > > > > On Thu, Dec 7, 2023 at 1:45 PM gongzhongqiang < > > gongzhongqi...@apache.org > > > > > > > > wrote: > > > > > > > > > It's very exciting to hear the news. > > > > > +1 for adding CDC Connectors to Apache Flink ! > > > > > > > > > > > > > > > Best, > > > > > Zhongqiang > > > > > > > > > > Leonard Xu <xbjt...@gmail.com> 于2023年12月7日周四 11:25写道： > > > > > > > > > > > Dear Flink devs, > > > > > > > > > > > > > > > > > > As you may have heard, we at Alibaba (Ververica) are planning to > > > donate > > > > > CDC Connectors for the Apache Flink project > > > > > > *[1]* to the Apache Flink community. > > > > > > > > > > > > > > > > > > > > > > > > CDC Connectors for Apache Flink comprise a collection of source > > > > > connectors designed specifically for Apache Flink. These connectors > > > > > > *[2]* > > > > > > enable the ingestion of changes from various databases using > > Change > > > > > Data Capture (CDC), most of these CDC connectors are powered by > > > Debezium > > > > > > *[3]* > > > > > > . They support both the DataStream API and the Table/SQL API, > > > > > facilitating the reading of database snapshots and continuous > reading > > > of > > > > > transaction logs with exactly-once processing, even in the event of > > > > > failures. > > > > > > > > > > > > > > > > > > > > > > > > Additionally, in the latest version 3.0, we have introduced many > > > > > long-awaited features. Starting from CDC version 3.0, we've built a > > > > > Streaming ELT Framework available for streaming data integration. > > This > > > > > framework allows users to write their data synchronization logic > in a > > > > > simple YAML file, which will automatically be translated into a > Flink > > > > > DataStreaming job. It emphasizes optimizing the task submission > > process > > > > and > > > > > offers advanced functionalities such as whole database > > synchronization, > > > > > merging sharded tables, and schema evolution > > > > > > *[4]*. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I believe this initiative is a perfect match for both sides. For > > the > > > > > Flink community, it presents an opportunity to enhance Flink's > > > > competitive > > > > > advantage in streaming data integration, promoting the healthy > growth > > > and > > > > > prosperity of the Apache Flink ecosystem. For the CDC Connectors > > > project, > > > > > becoming a sub-project of Apache Flink means being part of a > neutral > > > > > open-source community, which can attract a more diverse pool of > > > > > contributors. > > > > > > > > > > > > > > > > > > Please note that the aforementioned points represent only some of > > our > > > > > motivations and vision for this donation. Specific future > operations > > > need > > > > > to be further discussed in this thread. For example, the > sub-project > > > name > > > > > after the donation; we hope to name it Flink-CDC > > > > > > aiming to streaming data intergration through Apache Flink, > > > > > > following the naming convention of Flink-ML; And this project is > > > > managed > > > > > by a total of 8 maintainers, including 3 Flink PMC members and 1 > > Flink > > > > > Committer. The remaining 4 maintainers are also highly active > > > > contributors > > > > > to the Flink community, donating this project to the Flink > community > > > > > implies that their permissions might be reduced. Therefore, we may > > need > > > > to > > > > > bring up this topic for further discussion within the Flink PMC. > > > > > Additionally, we need to discuss how to migrate existing users and > > > > > documents. We have a user group of nearly 10,000 people and a > > > > multi-version > > > > > documentation site need to migrate. We also need to plan for the > > > > migration > > > > > of CI/CD processes and other specifics. > > > > > > > > > > > > > > > > > > > > > > > > While there are many intricate details that require > implementation, > > > we > > > > > are committed to progressing and finalizing this donation process. > > > > > > > > > > > > > > > > > > > > > > > > Despite being Flink’s most active ecological project (as > evaluated > > by > > > > > GitHub metrics), it also boasts a significant user base. However, I > > > > believe > > > > > it's essential to commence discussions on future operations only > > after > > > > the > > > > > community reaches a consensus on whether they desire this donation. > > > > > > > > > > > > > > > > > > Really looking forward to hear what you think! > > > > > > > > > > > > > > > > > > > > > > > > Best, > > > > > > Leonard (on behalf of the Flink CDC Connectors project > maintainers) > > > > > > > > > > > > [1] https://github.com/ververica/flink-cdc-connectors > > > > > > [2] > > > > > > > > > > > > > > > > > > > > > https://ververica.github.io/flink-cdc-connectors/master/content/overview/cdc-connectors.html > > > > > > [3] https://debezium.io > > > > > > [4] > > > > > > > > > > > > > > > > > > > > > https://ververica.github.io/flink-cdc-connectors/master/content/overview/cdc-pipeline.html > > > > > > > > > > > > > > > > > > > > > > > > -- > > > Best regards, > > > Sergey > > > > > >
