Hi, I second to Jark's suggestion to continue the discussion and task on mailing list or an issue.
Since the diff is clear from InLong side, a well described issue should be a good start point. May Charles or the author of these changes create one? Apart from that the QR code photo is broken on mailing list, I tend not to use such private channel. Basically, open-source collaboration happens in public and everyone is potential contributor. It's not we call several associated employees to finish jobs with a hard estimate. Best, tison. Charles Zhang <[email protected]> 于2022年7月24日周日 16:56写道： > Hi, tison and Jark, > Thanks for your advice. > Different from Flink CDC connectors, InLong's changes to MySQL CDC > connector mainly involve entire database migration and support for rich > metadata fields (like canal), which are very important for Apache InLong to > provide external services as data integration. For example, for the entire > database migration, if one schema is used for one table, it will be a waste > of resources and difficult to maintain, and it will also put pressure on > the upstream database.We also particularly hope to work with Flink CDC to > see how to support the entire database migration and support richer > metadata fields, so that we can directly rely on it. We don't know if Flink > CDC has plans and technical solutions for these two scenarios. We can talk > about it together or contribute directly. > In addition, in order to speed up discussion and follow-up > cooperation, I have created a WeChat group [1], and interested contributors > can join the direct discussion. > Thanks again. > [1] > [image: image.png] > > Jark Wu <[email protected]> 于2022年7月23日周六 22:45写道： > >> Thanks for starting this discussion, tison! >> >> It's great to see InLong is using flink-cdc-connectors to extract data >> from >> various databases. >> I think tison raised a good point about contributing back instead of >> forking. I believe this can >> help the InLong community in the long term by leveraging improvements and >> bug fixes from >> upstream. Flink CDC can also benefit from more contributions from the >> InLong community. >> I believe it's the only way that we can improve collaboration between >> different OSS projects. >> >> In a word, I highly appreciate learning the missing part of Flink CDC that >> InLong needs. >> We can discuss the missing features here or in the issues. >> >> Cheers, >> Jark >> >> >> On Fri, 22 Jul 2022 at 23:56, tison <[email protected]> wrote: >> >> > Hi, >> > >> > I notice that inlong-sort mysql-cdc connector[1] is almost copied from >> > Flink CDC mysql connector[2]. >> > >> > With a short discussion with Charles I learned that InLong developers >> > added metadata supports and migration of a whole database. >> > >> > It could be better to see if InLong community can contribute such >> supports >> > back to Flink CDC upstream, and avoid maintaining a whole fork as well >> as >> > periodically catch up upstream updates. >> > >> > Maintainers of Flink CDC projects are in cc (Leonard & Jark). >> > >> > Perhaps we can start with creating an issue on Flink CDC repo to >> describe >> > the case, contribute back the improvements, and switch dependency to the >> > upstream. >> > >> > Best, >> > tison. >> > >> > [1] >> > >> https://github.com/apache/inlong/tree/master/inlong-sort/sort-connectors/mysql-cdc >> > [2] >> > >> https://github.com/ververica/flink-cdc-connectors/tree/master/flink-connector-mysql-cdc >> > >> > >> > > > -- > Best wishes, > Charles Zhang >
