aliehsaeedii opened a new pull request, #22777: URL: https://github.com/apache/kafka/pull/22777
`StoredDescriptionTopologyEpoch` is the broker's record of which topology a Streams group's plugin entry holds (KIP-1331), and it drives whether the broker solicits a fresh push, whether the entry still needs deleting, and what describe serves. The plugin call and the metadata write that records the epoch are non-atomic, so a crash or error between them, or a push that races a delete, could leave the broker's belief and the plugin permanently disagreeing: a leak (the plugin holds a topology the broker thinks absent, so nothing reclaims it) or a loss (the plugin holds nothing but the broker thinks an epoch is stored, so it never re-solicits and describe reports NOT_STORED forever). This makes the epoch three-valued by adding a -2 "uncertain" marker beside the existing real epoch (>= 0, definitely held) and -1 (definitely empty). Uncertain is read like -1 for the solicitation decision and like a real epoch for the deletion decision, so a group left uncertain both re-solicits and stays reclaimable. Every operation that disturbs the plugin commits -2 durably before the plugin call and writes the final value on success; any failure after the barrier leaves the group uncertain and therefore self-healing. The barrier is applied to the push path, the periodic expiration cleanup, DeleteGroups, and the conversion of an empty Streams group to a classic group on a classic JoinGroup, which previously tombstoned the Streams metadata and orphaned the plugin entry. The cleanup path also closes the delete-versus-push race: `deleteTopology` and a concurrent `setTopology` have no ordering, so after a successful delete the broker re-checks the stored epoch and clears it to -1 only if it is still uncertain, otherwise writes -2 back to force a re-solicit because the raced push may have been removed. The classic-JoinGroup conversion detects the empty-Streams case during the group lookup the join already performs, so non-Streams and already-classic joins add no extra operation on the hot path. ### Testing The full `:group-coordinator` test suite passes (1,982 tests, 0 failures), including new tests covering the mark-then-delete-then-finalize ordering of the cleanup cycle, the revived-group drop-out on the barrier write, the uncertain-state solicitation and deletion decisions, and the deferred classic-JoinGroup conversion. JIRA: https://issues.apache.org/jira/browse/KAFKA-20748 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
