lianetm commented on code in PR #18737:
URL: https://github.com/apache/kafka/pull/18737#discussion_r1945890862
##########
clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java:
##########
@@ -324,7 +323,7 @@ public CompletableFuture<Void>
maybeAutoCommitSyncBeforeRevocation(final long de
CompletableFuture<Void> result = new CompletableFuture<>();
OffsetCommitRequestState requestState =
- createOffsetCommitRequest(subscriptions.allConsumed(), deadlineMs);
+ createOffsetCommitRequest(latestPartitionOffsets, deadlineMs);
Review Comment:
More brainstorming improving this idea a bit more, all the above is
important if we have auto-commit enabled only. Also it's important to keep
resolving the topics names for the topic Ids assigned asap (to request the
metadata needed, that part doesn't need to wait for any in-flight fetch), so
even though we the need the above comment, we shouldn't throw away completely
the attempt to reconcile on the membershipMgr.poll.
So one option that comes to mind is to keep calling
`maybeReconcile(canCommit=false)` from the membershipMgr.poll to not delay the
resolution of the topics, and also not delay reconciliation if autoCommit
disabled (the new param is simply to short-circuit right after resolving
metadata, before `markReconciliationInProgress` I guess) if autoCommitEnabled &
!canCommit. Then we also need a call to `maybeReconcile(canCommit=true)` when a
PollEvent is processed (this is what the above comment is about, basically the
one to reconcile with autoCommit, only possible when
> we know any record returned to the application will have been processed
and no more records can be given to the application. So, it's safe to commit
the offset at that point.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
