SEZ9 commented on issue #11847: URL: https://github.com/apache/seatunnel/issues/11847#issuecomment-5358316360
Thanks @zhang-arvin for volunteering — assigned to you, please go ahead. To recap the scope so we stay focused: the problem is that a committed-offset restore can permanently skip a row when it commits between the savepoint's offset commit and the stop, because the replication slot's confirmed LSN advances past that row's commit LSN without the row ever being emitted. @davidzollo's latest data point reinforces that this is a dev-level timing race rather than something introduced by a single PR: the same `PostgresCDCIT.testPostgresCdcSnapshotOnlyAndCommittedOffsetStartupModes` failure reproduced on an unrelated engine-side branch (fork run 32094650264, job `all-connectors-it-6 (8, ubuntu-latest)`), with the symptom appearing since the `163a36ac..da7b1e01` window while dev's own CI passes intermittently. As he notes, intermittent green is the worst case here — the failure mode is silent row loss. Concrete asks for @zhang-arvin: 1. Try to reproduce the race locally, ideally with a deterministic hook or injected delay between the offset commit during savepoint and the actual stop, so we can confirm the confirmed-LSN-advances-past-uncommitted-emission sequence. 2. Once confirmed, share your analysis of where the fix belongs (deferring the slot's confirmed LSN advance vs. tightening the commit/stop ordering) before the PR, so we can align on the approach. 3. Please add or extend a test that fails reliably under the injected timing, not just the existing intermittent IT — otherwise we can't distinguish a real fix from a lucky green run. Please post your reproduction findings here when you have them. Thanks both! <!-- streview-comment:394 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
