Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

dajac merged PR #14557: URL: https://github.com/apache/kafka/pull/14557 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

dajac commented on PR #14557: URL: https://github.com/apache/kafka/pull/14557#issuecomment-1866401989 We've got a reasonably good build here: https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fkafka-pr/detail/PR-14557/34/tests/. Based on this one, it is safe to merge this PR. I

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

lianetm commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1432774544 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/events/ApplicationEventProcessor.java: ## @@ -272,6 +280,10 @@ private void process(final LeaveOnCl

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

lianetm commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1432723702 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/events/ApplicationEventProcessor.java: ## @@ -272,6 +280,10 @@ private void process(final LeaveOnCl

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

dajac commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1432722759 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/events/ApplicationEventProcessor.java: ## @@ -272,6 +280,10 @@ private void process(final LeaveOnClos

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

AndrewJSchofield commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1432721019 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/events/ApplicationEventProcessor.java: ## @@ -272,6 +280,10 @@ private void process(final

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

dajac commented on PR #14557: URL: https://github.com/apache/kafka/pull/14557#issuecomment-1864471696 > * [KAFKA-16033](https://issues.apache.org/jira/browse/KAFKA-16033): To review the retry logic, considering moving it to the callers. This definitely aligns with the legacy coordinator app

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

lianetm commented on PR #14557: URL: https://github.com/apache/kafka/pull/14557#issuecomment-1863193316 Thanks for the comments @dajac , all addressed. I also included [2248b55](https://github.com/apache/kafka/pull/14557/commits/2248b55505a49c04868fd7b5807599d14a74df73) with a minor fix and

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

lianetm commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1431620008 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumerTest.java: ## @@ -403,18 +402,13 @@ public void testEnsureCallbackExecutedByAppl

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

lianetm commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1431498965 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImplTest.java: ## @@ -262,6 +328,7 @@ public void testFencingWhenStateIsReconcilin

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

lianetm commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1431496640 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -190,63 +183,95 @@ private static long findMinTime(final Collectio

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

dajac commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1431151481 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/CommitRequestManagerTest.java: ## @@ -187,38 +192,47 @@ public void testPoll_EnsureCorrectInflightRe

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

lianetm commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1429362412 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -1119,8 +1125,11 @@ public void commitSync(Map offsets, Duration

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

lianetm commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1429362412 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -1119,8 +1125,11 @@ public void commitSync(Map offsets, Duration

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

lianetm commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1427097160 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -190,63 +182,91 @@ private static long findMinTime(final Collectio

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

lianetm commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1427097160 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -190,63 +182,91 @@ private static long findMinTime(final Collectio

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

lianetm commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1427026722 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -625,7 +627,9 @@ public void commitAsync(Map offsets, OffsetCo

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

lianetm commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1427025116 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -381,23 +441,37 @@ public NetworkClientDelegate.UnsentRequest toU

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

lianetm commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1427015080 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -190,63 +182,91 @@ private static long findMinTime(final Collectio

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

lianetm commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1426950983 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -190,63 +182,91 @@ private static long findMinTime(final Collectio

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

lianetm commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1426950983 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -190,63 +182,91 @@ private static long findMinTime(final Collectio

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

lianetm commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1426950983 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -190,63 +182,91 @@ private static long findMinTime(final Collectio

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

lianetm commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1426894017 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumerTest.java: ## @@ -463,35 +467,34 @@ public void testEnsurePollExecutedCommitAsync

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

lianetm commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1426889550 ## clients/src/test/java/org/apache/kafka/clients/admin/AdminClientTestUtils.java: ## @@ -33,6 +27,12 @@ import org.apache.kafka.common.config.ConfigResource; import

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

lianetm commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1426872271 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -780,6 +981,28 @@ List drain(final long currentTimeMs) {

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

lianetm commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1426857957 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -457,120 +539,224 @@ public void onResponse(final ClientResponse r

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

lianetm commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1426846736 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -429,21 +492,40 @@ public void onResponse(final ClientResponse res

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

lianetm commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1426842174 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -457,120 +539,224 @@ public void onResponse(final ClientResponse r

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

dajac commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1426802152 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -780,6 +981,28 @@ List drain(final long currentTimeMs) {

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

lianetm commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1426789748 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -429,21 +492,40 @@ public void onResponse(final ClientResponse res

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

dajac commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1426782966 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -429,21 +492,40 @@ public void onResponse(final ClientResponse respo

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

dajac commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1426753201 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -625,7 +627,9 @@ public void commitAsync(Map offsets, OffsetCo }

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

dajac commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1426746530 ## clients/src/test/java/org/apache/kafka/clients/admin/AdminClientTestUtils.java: ## @@ -33,6 +27,12 @@ import org.apache.kafka.common.config.ConfigResource; import or

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

dajac commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1426743136 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -457,120 +539,224 @@ public void onResponse(final ClientResponse res

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

dajac commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1426740860 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -429,21 +492,40 @@ public void onResponse(final ClientResponse respo

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

dajac commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1426740281 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -429,21 +492,40 @@ public void onResponse(final ClientResponse respo

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

dajac commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1426737762 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -457,120 +539,224 @@ public void onResponse(final ClientResponse res

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

dajac commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1426733887 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -381,23 +441,37 @@ public NetworkClientDelegate.UnsentRequest toUns

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

dajac commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1426727298 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -190,63 +182,91 @@ private static long findMinTime(final Collection

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

dajac commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1426720259 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -1119,8 +1125,11 @@ public void commitSync(Map offsets, Duration

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

cadonna commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1426483385 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumerTest.java: ## @@ -463,35 +467,34 @@ public void testEnsurePollExecutedCommitAsync

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

cadonna commented on PR #14557: URL: https://github.com/apache/kafka/pull/14557#issuecomment-1855434350 There are compile errors in the builds. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

lianetm commented on PR #14557: URL: https://github.com/apache/kafka/pull/14557#issuecomment-1854868531 Thanks for the review! All comments addressed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

lianetm commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1425995440 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -741,61 +767,86 @@ boolean reconcile() { revoked

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

lianetm commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1425984924 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -1077,4 +1131,18 @@ public void onUpdate(ClusterResource clusterR

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

lianetm commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1425982023 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -190,63 +179,90 @@ private static long findMinTime(final Collectio

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

lianetm commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1425965837 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -457,120 +535,224 @@ public void onResponse(final ClientResponse r

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

lianetm commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1425958312 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/events/CommitApplicationEvent.java: ## @@ -18,17 +18,36 @@ import org.apache.kafka.clients.consu

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

lucasbru commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1425393938 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -1104,8 +1137,21 @@ private void clearPendingAssignmentsAndLocal

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

lianetm commented on code in PR #14557: URL: https://github.com/apache/kafka/pull/14557#discussion_r1423159520 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -619,11 +838,36 @@ private CompletableFuture> chainFuture(fi

Re: [PR] KAFKA-15456: Commit/Fetch error handling improvements and V9 support [kafka]

lianetm commented on PR #14557: URL: https://github.com/apache/kafka/pull/14557#issuecomment-1847909912 hey @dajac, this is ready for review now now, repurposed to include general improvement for the commit/fetch, including support for the new v9 errors too. Thanks! -- This is an automat