SEZ9 commented on issue #13280: URL: https://github.com/apache/seatunnel/issues/13280#issuecomment-6887812137
Thanks for the detailed, deterministic reproduction and the code investigation. If I follow correctly: `SqlServerStreamingChangeEventSource` calls `afterHandleLsn(partition, offsetContext.getOffset())` after querying changes, and `SqlServerTransactionLogFetchTask.TransactionLogSplitReadTask.afterHandleLsn` compares that row-event offset with `lsnSplit.getStopOffset()` before emitting END. Since HIGH is database-wide, when only `Other` advances it and the selected table `A` has no changes in the LOW-HIGH interval, the offset of A's last event can never reach HIGH, so END is not emitted and the buffered snapshot records are never released. That matches the missing `backfillReadTask execute end` line in your log. Your local fix (capping each bounded query at the snapshot HIGH watermark and emitting END at the original HIGH once the complete query results, including an empty result, have been processed) sounds like a reasonable direction. Would you be willing to open a PR with that change, along with an integration test covering the "another table advances HIGH while the selected table has no changes" scenario? Your reproduction steps are a good basis for it. If you would rather not, please let us know and we can pick it up. <!-- streview-comment:2355 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
