SEZ9 commented on issue #12624:
URL: https://github.com/apache/seatunnel/issues/12624#issuecomment-6050930618
@xinnyuli Thanks for confirming 1(b) — that is the real gap. Answering your
two questions:
**Should a rollback with pending batches make later commits on that
connection fail?** Yes. A full `rollback()` ends the JDBC transaction, so it is
fine for it to clear the "work pending on this transaction" fact. But if that
rollback discarded any flushed batch, it must also mark the shared checkpoint
interval as having lost work, so that a later `prepareCommit` or close-time
commit from another table on the same queue index fails instead of reporting
success. Otherwise table B commits cleanly on the same (or a replacement)
connection after table A's rows are gone, which is exactly the silent partial
commit this issue is about. So the shared state needs two things: the identity
of the connection that owns flushed work, and a "this interval already lost
work" flag. A successful shared commit is the only normal path that clears both
for every writer; a savepoint rollback must leave the pre-savepoint state
visible.
**Same PR or follow-up?** Same PR, please. A per-`JdbcOutputFormat` marker
cannot protect a transaction shared by multiple writers, so moving the state to
the queue/connection owner in `ConnectionPoolManager` (or an equivalent object
shared by all writers for that queue index) is part of the fix rather than an
enhancement. Please update the known-limitation note only once this is closed.
Concrete asks:
1. Queue-scoped transaction state as described above, exposed through the
provider so every writer on that queue sees it.
2. The guard must cover every commit exit, including the `close()` path, so
it goes through the same `commitIfNeeded()` check rather than calling
`connection.commit()` directly.
3. Deterministic tests with two tables on the same queue index:
- A flushes a manual-commit batch, a full rollback happens on the shared
connection; B's `prepareCommit` and `close` both fail.
- The shared connection dies and is replaced after A flushes; B must not
commit on the replacement.
- A savepoint rollback keeps the pre-savepoint state visible; a
successful shared commit clears it for all writers.
Ping me when that's pushed and I'll take another look.
<!-- streview-comment:1587 -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
