davidzollo commented on PR #10808: URL: https://github.com/apache/seatunnel/pull/10808#issuecomment-5337618161
Updated this PR with commit `b1a3d64aa1b6e65a3699f72d6833a37f587726d2` to address the latest enumerator/checkpoint concurrency blockers. What changed: - `enumerator.run()` no longer holds the `enumeratorContext` checkpoint monitor, so long-running streaming enumerators can no longer block checkpoint barriers for their full lifetime. - split delivery no longer synchronizes on `enumeratorContext`, removing the connector-state-lock -> enumerator-context / enumerator-context -> connector-state-lock AB/BA path. - checkpoint snapshot now opens a short reader-barrier gate after `snapshotState()`, waits for any split delivery registered during the snapshot, and only then forwards the reader barrier. New deliveries that arrive after the snapshot wait until the reader barrier has been sent. - the delivery placeholder is registered under the same gate lock used by the reader-barrier phase, closing the gate-check / delivery-registration TOCTOU window. - added focused regression coverage for long-running `run()` plus checkpoint, split delivery without the enumerator-context monitor, and delivery ack ordering before reader barrier. Local validation, within the SeaTunnel local-check boundary used for this PR: `./mvnw -nsu -DskipTests -DskipIT=true -Dmaven.gitcommitid.skip=true -pl seatunnel-engine/seatunnel-engine-server -am spotless:apply spotless:check` Result: `BUILD SUCCESS`. The PR is now ready for human re-review while the new current-head GitHub checks run. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
