Thank you, Gunnar! I have created a draft PR for the proposed feature - https://github.com/apache/flink-cdc/pull/4170 Looking forward to hearing more feedback.
Thank you, Tejansh From: Gunnar Morling <[email protected]> Date: Monday, 3 November 2025 at 17:19 To: [email protected] <[email protected]> Subject: Re: [PROPOSAL] Support for MySQL Transaction Boundary Events in Flink CDC Connector [You don't often get email from [email protected]. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ] EXTERNAL EMAIL : Do not click any links or open any attachments unless you trust the sender and know the content is safe. Hey all, I'd love to see support for this! Coincidentally, I am just working on a PoC right now which uses the custom watermarks in the DataStream v2 API to represent transaction boundaries. It seems this is a great fit conceptually. In any case, it would be nice to not only support this for MySQL but also other DBs. Debezium provides that transaction metadata for a range of connectors, including Postgres. --Gunnar On Mon, 3 Nov 2025 at 15:53, Tejansh Rana <[email protected]> wrote: > Hello, > > Following up on the below proposal. Would appreciate your thoughts and if > we could move forward with this feature. > > Thank you, > Tejansh > > From: Tejansh Rana <[email protected]> > Date: Friday, 17 October 2025 at 15:58 > To: [email protected] <[email protected]> > Subject: [PROPOSAL] Support for MySQL Transaction Boundary Events in Flink > CDC Connector > You don't often get email from [email protected]. Learn > why this is important<https://aka.ms/LearnAboutSenderIdentification> > > EXTERNAL EMAIL : Do not click any links or open any attachments unless you > trust the sender and know the content is safe. > > Hi team, > > Following my discussion with Leonard Xu at Flink Forward, I am writing to > propose a feature enhancement for the Flink MySQL CDC connector related to > how it handles transaction metadata from the MySQL binary log. > > Problem Statement: > In data streaming pipelines that require transactional guarantees or need > to group atomic changes together, it is essential to identify the > boundaries of the original database transaction (i.e., the BEGIN and COMMIT > or END events). Currently, the Flink MySQL CDC connector appears to skip > these transaction lifecycle events - > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fflink-cdc%2Fblob%2F23a1c2efb6fa9ce1c9f17b3836f6aaa995bb0660%2Fflink-cdc-connect%2Fflink-cdc-source-connectors%2Fflink-connector-mysql-cdc%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fflink%2Fcdc%2Fconnectors%2Fmysql%2Fsource%2Freader%2FMySqlRecordEmitter.java%23L77&data=05%7C02%7Ctejansh.rana%40autodesk.com%7C24d0f6303e9f49a16dc908de1afd3461%7C67bff79e7f914433a8e5c9252d2ddc1d%7C0%7C0%7C638977871962762959%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=9KT7qKicpHoHobqCjYrjsQ%2FmEDh0WAuO7ahY1X%2FhV7o%3D&reserved=0<https://github.com/apache/flink-cdc/blob/23a1c2efb6fa9ce1c9f17b3836f6aaa995bb0660/flink-cdc-connect/flink-cdc-source-connectors/flink-connector-mysql-cdc/src/main/java/org/apache/flink/cdc/connectors/mysql/source/reader/MySqlRecordEmitter.java#L77> > . > I have also attached a screenshot of the logs from this behaviour. > > This omission makes it challenging to reconstruct the original transaction > scope. Without explicit transaction markers, downstream Flink jobs cannot > easily guarantee atomicity across sinks. > > Proposed Solution: > The underlying CDC mechanism, Debezium, supports emitting transaction > boundary events (BEGIN and END/COMMIT) through its configuration. > > We propose enhancing the Flink MySQL CDC connector to expose this > transaction metadata to the Flink pipeline. The connector should emit > specialised records or metadata fields that indicate the start and end of a > transaction as emitted. We would be happy to create a PR with this feature > if this proposal goes ahead. > > Thank you, > Tejansh > > >
