WXPNG opened a new issue, #12565: URL: https://github.com/apache/seatunnel/issues/12565
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened When a CDC job (built on connector-cdc-base, e.g. MySQL-CDC) is running in incremental mode with exactly-once, and I stop it with a savepoint, add a new table to the capture list, then restart it from the previous job id / restore from checkpoint, the pre-existing tables (already in the pure-increment phase) silently lose incremental changes. The incremental split after restore mixes two kinds of tables: old tables that finished snapshot in a previous run, and new tables snapshotted in the current run. But the split start offset and the per-table emission filter only account for the new tables, so changes produced on the old tables between the savepoint offset and the new table's snapshot watermark are dropped. ### SeaTunnel Version dev ### SeaTunnel Config ```conf env { parallelism = 1 job.mode = "STREAMING" checkpoint.interval = 5000 } source { MySQL-CDC { base-url = "jdbc:mysql://localhost:3306/test" username = "root" password = "xxxxxx" table-names = ["test.t1", "test.t2"] startup.mode = "initial" exactly_once = true } } sink { Console {} } ``` ### Running Command ```shell # Create replication job (jobID: 1152905286592233473) curl -X POST "http://127.0.0.1:8080/submit-job?format=hocon"; \ -H "Content-Type: text/plain" \ -d "$(cat mysql_2_xxxx.conf)" # Stop with savepoint curl -X POST "http://127.0.0.1:8080/stop-job"; \ -H "Content-Type: application/json" \ -d '{ "jobId": 1152905286592233473, "isStopWithSavePoint": true, "force": false }' # Add t3 to table-names # Restart with savepoint curl -X POST "http://127.0.0.1:8080/submit-job?format=hocon&jobId= 1152905286592233473&isStartWithSavePoint=true" \ -H "Content-Type: text/plain" \ -d "$(cat mysql_2_xxxx.conf)" ``` ### Error Exception ```log No exception is thrown. The symptom is silent data loss: the incremental changes of the old tables (t1/t2) produced during the restore window are missing in the target, while the new table (t3) is synchronized normally. ``` ### Zeta or Flink or Spark Version Zeta ### Java or Scala Version JDK 8 ### Screenshots (optional: a comparison showing that t1/t2 incremental records during the restore window are missing in the target while t3 is present) ### Are you willing to submit PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
