JNSimba opened a new pull request, #66998:
URL: https://github.com/apache/doris/pull/66998
### What problem does this PR solve?
Issue Number: N/A
Related PR: #66409
Problem Summary:
When a non-GTID MySQL transaction is being delivered and the event listener
is blocked by backpressure, the keepalive thread can reconnect after the
in-memory binlog position has advanced into the transaction. The reconnect then
resumes from the middle of the transaction and can silently skip rows.
This change tracks the start of an incomplete non-GTID transaction and
rewinds to that position before a keepalive reconnect. Completed transactions
clear the replay state. It also separates the CDC progress heartbeat from the
connection keepalive interval and uses the configured progress heartbeat as the
MySQL protocol heartbeat.
The Failsafe test classpath now uses the compiled classes directory so
integration tests exercise locally overridden dependency classes instead of the
repackaged application artifact.
### Release note
Fix potential MySQL CDC data loss when keepalive reconnects during an
incomplete non-GTID transaction.
### Check List (For Author)
- Test
- [x] Unit Test
- [x] Manual test
- Built with `build.sh --be-cdc-client`.
- Transaction replay unit tests: 8 passed.
- Keepalive reconnect integration test: passed 3 consecutive runs.
- Existing MySQL basic-read and PostgreSQL stream-read integration
tests passed.
- Behavior changed:
- [x] Yes. Incomplete non-GTID transactions are replayed from their
start after keepalive reconnect.
- Does this need documentation?
- [x] No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
