emlynazuma commented on PR #17376:
URL: https://github.com/apache/iceberg/pull/17376#issuecomment-5115396327
A few suggestions now that this has grown quite a bit, meant to help it land
faster rather than add
scope:
**Split the election rework into its own PR.** The fix for the actual
reported bug (#16282 —
duplicate files) doesn't depend on how leadership is elected:
- offset-filter dedup in `consumeAvailable`
- fixed `transactional.id` fencing
- `earliest` reset for coordinator recovery
- retryable-commit-error classification
The `open()`/`close()` election rewrite (dropping the Admin dependency) is a
separate, larger
architectural change with a much bigger surface. Splitting it out would let
the duplicate-file fix
land on its own — smaller, more urgent, easier for a committer to review —
while the election rework
gets its own PR with more room for scrutiny.
**A couple of things worth resolving before requesting review:**
- `kafka-connect-tests` is currently failing (4/13 integration tests, all
timing out waiting for a
snapshot within 30s). Might be worth checking whether `open()`'s one-shot
`subscription()` read has
a path where it comes back incomplete and nothing retries the election,
now that the
`save()`-level reconcile was dropped.
- The branch has 39 commits, including some unrelated reverted work and ~24
merge-main commits —
squashing onto a clean branch would make it much easier to follow what
changed and why.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
