Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-14 Thread via GitHub
jlprat commented on PR #16310: URL: https://github.com/apache/kafka/pull/16310#issuecomment-2169158166 Hi @chia7712 we can let it stay in the 3.8 branch -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-14 Thread via GitHub
chia7712 commented on PR #16310: URL: https://github.com/apache/kafka/pull/16310#issuecomment-2169076981 > Can this get merged into 3.8.0? Sorry that I did not wait for @jlprat feedback before backporting to 3.8 ... @jlprat I will revert that if it is unsuitable to be in 3.8 :)

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-14 Thread via GitHub
kirktrue commented on PR #16310: URL: https://github.com/apache/kafka/pull/16310#issuecomment-2169044767 @jlprat Can this get merged into 3.8.0? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-14 Thread via GitHub
chia7712 merged PR #16310: URL: https://github.com/apache/kafka/pull/16310 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-14 Thread via GitHub
kirktrue commented on PR #16310: URL: https://github.com/apache/kafka/pull/16310#issuecomment-2168568698 @AndrewJSchofield @cadonna @chia7712 @lianetm @philipnee: this PR is ready to be re-reviewed. Thanks all for your continued input šŸ˜„ -- This is an automated message from the Apache Git

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-14 Thread via GitHub
kirktrue commented on code in PR #16310: URL: https://github.com/apache/kafka/pull/16310#discussion_r1640213245 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -1666,23 +1668,68 @@ private boolean initWithCommittedOffsetsIfNeed

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-14 Thread via GitHub
kirktrue commented on code in PR #16310: URL: https://github.com/apache/kafka/pull/16310#discussion_r1640209483 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -1666,23 +1668,68 @@ private boolean initWithCommittedOffsetsIfNeed

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-14 Thread via GitHub
kirktrue commented on code in PR #16310: URL: https://github.com/apache/kafka/pull/16310#discussion_r1640117064 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -1666,23 +1668,68 @@ private boolean initWithCommittedOffsetsIfNeed

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-14 Thread via GitHub
kirktrue commented on code in PR #16310: URL: https://github.com/apache/kafka/pull/16310#discussion_r1640115221 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -1666,23 +1668,68 @@ private boolean initWithCommittedOffsetsIfNeed

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-14 Thread via GitHub
chia7712 commented on code in PR #16310: URL: https://github.com/apache/kafka/pull/16310#discussion_r1640111560 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -1666,23 +1668,68 @@ private boolean initWithCommittedOffsetsIfNeed

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-14 Thread via GitHub
kirktrue commented on code in PR #16310: URL: https://github.com/apache/kafka/pull/16310#discussion_r1640101706 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -1666,23 +1668,68 @@ private boolean initWithCommittedOffsetsIfNeed

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-14 Thread via GitHub
cadonna commented on code in PR #16310: URL: https://github.com/apache/kafka/pull/16310#discussion_r1639493007 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -1666,23 +1668,68 @@ private boolean initWithCommittedOffsetsIfNeede

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-14 Thread via GitHub
jlprat commented on PR #16310: URL: https://github.com/apache/kafka/pull/16310#issuecomment-2167378623 Hi @kirktrue. Let's check again when the PR is approved :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-13 Thread via GitHub
chia7712 commented on code in PR #16310: URL: https://github.com/apache/kafka/pull/16310#discussion_r1639296989 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -1666,23 +1668,68 @@ private boolean initWithCommittedOffsetsIfNeed

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-13 Thread via GitHub
kirktrue commented on PR #16310: URL: https://github.com/apache/kafka/pull/16310#issuecomment-2167013423 @jlpratā€”This Jira/PR is a blocker for the KIP-848 Java client work. It sounds like we're really close to merging this within the next day or two. Thanks! -- This is an automated messa

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-13 Thread via GitHub
kirktrue commented on PR #16241: URL: https://github.com/apache/kafka/pull/16241#issuecomment-2166953380 Closing this PR as we have achieved consensus around the simpler solution in #16310. -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-13 Thread via GitHub
kirktrue closed pull request #16241: KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively URL: https://github.com/apache/kafka/pull/16241 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-13 Thread via GitHub
kirktrue commented on code in PR #16310: URL: https://github.com/apache/kafka/pull/16310#discussion_r1639005027 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumerTest.java: ## @@ -574,6 +581,148 @@ public void testPollLongThrowsException() {

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-13 Thread via GitHub
kirktrue commented on code in PR #16310: URL: https://github.com/apache/kafka/pull/16310#discussion_r1639004424 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumerTest.java: ## @@ -574,6 +575,93 @@ public void testPollLongThrowsException() {

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-13 Thread via GitHub
kirktrue commented on code in PR #16310: URL: https://github.com/apache/kafka/pull/16310#discussion_r1639004097 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumerTest.java: ## @@ -574,6 +575,93 @@ public void testPollLongThrowsException() {

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-13 Thread via GitHub
kirktrue commented on PR #16310: URL: https://github.com/apache/kafka/pull/16310#issuecomment-2166875819 @AndrewJSchofield @cadonna @lianetm @philipnee: this PR is ready to be re-reviewed. Thanks all for your input šŸ˜„ -- This is an automated message from the Apache Git Service. To respond

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-13 Thread via GitHub
kirktrue commented on code in PR #16310: URL: https://github.com/apache/kafka/pull/16310#discussion_r1638998663 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumerTest.java: ## @@ -574,6 +581,148 @@ public void testPollLongThrowsException() {

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-13 Thread via GitHub
kirktrue commented on code in PR #16310: URL: https://github.com/apache/kafka/pull/16310#discussion_r1638996787 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -1666,13 +1668,28 @@ private boolean initWithCommittedOffsetsIfNeed

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-13 Thread via GitHub
kirktrue commented on code in PR #16310: URL: https://github.com/apache/kafka/pull/16310#discussion_r1638982381 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumerTest.java: ## @@ -574,6 +581,148 @@ public void testPollLongThrowsException() {

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-13 Thread via GitHub
lianetm commented on code in PR #16310: URL: https://github.com/apache/kafka/pull/16310#discussion_r1638974414 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumerTest.java: ## @@ -574,6 +581,148 @@ public void testPollLongThrowsException() {

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-13 Thread via GitHub
lianetm commented on code in PR #16310: URL: https://github.com/apache/kafka/pull/16310#discussion_r1638974414 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumerTest.java: ## @@ -574,6 +581,148 @@ public void testPollLongThrowsException() {

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-13 Thread via GitHub
kirktrue commented on code in PR #16310: URL: https://github.com/apache/kafka/pull/16310#discussion_r1638953458 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumerTest.java: ## @@ -574,6 +581,148 @@ public void testPollLongThrowsException() {

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-13 Thread via GitHub
lianetm commented on code in PR #16310: URL: https://github.com/apache/kafka/pull/16310#discussion_r1638948285 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumerTest.java: ## @@ -574,6 +581,148 @@ public void testPollLongThrowsException() {

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-13 Thread via GitHub
lianetm commented on PR #16310: URL: https://github.com/apache/kafka/pull/16310#issuecomment-2166791581 Thanks for the updates @kirktrue , happy with the approach too. Left an answer to your concerns on the tests, can be totally done separately if you prefer. -- This is an automated mess

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-13 Thread via GitHub
lianetm commented on code in PR #16310: URL: https://github.com/apache/kafka/pull/16310#discussion_r1638945377 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumerTest.java: ## @@ -574,6 +581,148 @@ public void testPollLongThrowsException() {

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-13 Thread via GitHub
AndrewJSchofield commented on PR #16310: URL: https://github.com/apache/kafka/pull/16310#issuecomment-2166720052 I'm happy with the approach now. Thanks @kirktrue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-13 Thread via GitHub
AndrewJSchofield commented on code in PR #16310: URL: https://github.com/apache/kafka/pull/16310#discussion_r1638839026 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -1666,13 +1668,28 @@ private boolean initWithCommittedOffse

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-13 Thread via GitHub
kirktrue commented on code in PR #16310: URL: https://github.com/apache/kafka/pull/16310#discussion_r1638837576 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumerTest.java: ## @@ -574,6 +581,148 @@ public void testPollLongThrowsException() {

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-13 Thread via GitHub
AndrewJSchofield commented on code in PR #16310: URL: https://github.com/apache/kafka/pull/16310#discussion_r1638837517 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumerTest.java: ## @@ -574,6 +581,148 @@ public void testPollLongThrowsExcept

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-13 Thread via GitHub
kirktrue commented on code in PR #16310: URL: https://github.com/apache/kafka/pull/16310#discussion_r1638836529 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumerTest.java: ## @@ -574,6 +581,148 @@ public void testPollLongThrowsException() {

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-13 Thread via GitHub
kirktrue commented on code in PR #16310: URL: https://github.com/apache/kafka/pull/16310#discussion_r1638835624 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -1666,13 +1668,28 @@ private boolean initWithCommittedOffsetsIfNeed

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-13 Thread via GitHub
kirktrue commented on code in PR #16310: URL: https://github.com/apache/kafka/pull/16310#discussion_r1638821598 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -1666,13 +1668,28 @@ private boolean initWithCommittedOffsetsIfNeed

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-13 Thread via GitHub
lianetm commented on code in PR #16310: URL: https://github.com/apache/kafka/pull/16310#discussion_r1638194106 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumerTest.java: ## @@ -574,6 +575,93 @@ public void testPollLongThrowsException() {

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-13 Thread via GitHub
lianetm commented on code in PR #16310: URL: https://github.com/apache/kafka/pull/16310#discussion_r1638194106 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumerTest.java: ## @@ -574,6 +575,93 @@ public void testPollLongThrowsException() {

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-13 Thread via GitHub
lianetm commented on PR #16310: URL: https://github.com/apache/kafka/pull/16310#issuecomment-2165679908 Thanks for the PR @kirktrue, I love the simple the approach and the consistency with the legacy logic, thanks for your patience here :) Same as @cadonna , LGTM once the error handli

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-13 Thread via GitHub
lianetm commented on code in PR #16310: URL: https://github.com/apache/kafka/pull/16310#discussion_r1638203661 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumerTest.java: ## @@ -574,6 +575,93 @@ public void testPollLongThrowsException() {

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-13 Thread via GitHub
lianetm commented on code in PR #16310: URL: https://github.com/apache/kafka/pull/16310#discussion_r1638194106 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumerTest.java: ## @@ -574,6 +575,93 @@ public void testPollLongThrowsException() {

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-13 Thread via GitHub
lianetm commented on code in PR #16310: URL: https://github.com/apache/kafka/pull/16310#discussion_r1638194106 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumerTest.java: ## @@ -574,6 +575,93 @@ public void testPollLongThrowsException() {

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-13 Thread via GitHub
lianetm commented on code in PR #16310: URL: https://github.com/apache/kafka/pull/16310#discussion_r1638194106 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumerTest.java: ## @@ -574,6 +575,93 @@ public void testPollLongThrowsException() {

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-13 Thread via GitHub
lianetm commented on code in PR #16310: URL: https://github.com/apache/kafka/pull/16310#discussion_r1638183905 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -1666,13 +1668,28 @@ private boolean initWithCommittedOffsetsIfNeede

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-13 Thread via GitHub
AndrewJSchofield commented on code in PR #16310: URL: https://github.com/apache/kafka/pull/16310#discussion_r1637780052 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -1666,13 +1668,28 @@ private boolean initWithCommittedOffse

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-12 Thread via GitHub
kirktrue commented on PR #16310: URL: https://github.com/apache/kafka/pull/16310#issuecomment-2164238647 @AndrewJSchofield @cadonna @lianetm @philipneeā€”please review this PR. It's an alternative take on #16241 that seems simpler and not fraught with peril. Thanks! -- This is an aut

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-12 Thread via GitHub
kirktrue commented on code in PR #16310: URL: https://github.com/apache/kafka/pull/16310#discussion_r1637405808 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -1983,6 +2009,10 @@ SubscriptionState subscriptions() { re

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-12 Thread via GitHub
philipnee commented on code in PR #16310: URL: https://github.com/apache/kafka/pull/16310#discussion_r1637370335 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java: ## @@ -1983,6 +2009,10 @@ SubscriptionState subscriptions() { r

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-12 Thread via GitHub
kirktrue commented on PR #16241: URL: https://github.com/apache/kafka/pull/16241#issuecomment-2164161606 @lianetm @cadonna @AndrewJSchofieldā€”PTAL at an alternate implementation of this Jira: #16310. Thanks! -- This is an automated message from the Apache Git Service. To respond to the mes

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-12 Thread via GitHub
lianetm commented on PR #16241: URL: https://github.com/apache/kafka/pull/16241#issuecomment-2163819417 Hey @kirktrue, I don't expect the code snippets you shared in attempt1 would work because in the end it creates a new event every time. The approach I had in mind is more like what

[PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-12 Thread via GitHub
kirktrue opened a new pull request, #16310: URL: https://github.com/apache/kafka/pull/16310 Allow the committed offsets fetch to run for as long as needed. This handles the case where a user invokes `Consumer.poll()` with a very small timeout (including zero). ### Committer Checklist

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-12 Thread via GitHub
kirktrue commented on PR #16241: URL: https://github.com/apache/kafka/pull/16241#issuecomment-2163677488 @lianetmā€”thanks for your suggestion for handling it at the `AsyncKafkaConsumer` layer. > But what about just moving the timeout to the get result? > > 1. issue fetch (max_va

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-12 Thread via GitHub
kirktrue commented on code in PR #16241: URL: https://github.com/apache/kafka/pull/16241#discussion_r1636897803 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -519,6 +524,7 @@ private void fetchOffsetsWithRetries(final Offs

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-12 Thread via GitHub
kirktrue commented on code in PR #16241: URL: https://github.com/apache/kafka/pull/16241#discussion_r1636897052 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -1279,4 +1288,119 @@ static class MemberInfo { this.

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-11 Thread via GitHub
lianetm commented on PR #16241: URL: https://github.com/apache/kafka/pull/16241#issuecomment-2161273163 High level comment about the approach. Getting into caching results of fetch requests seems very tricky, given how easily they can become stale. The simple dedup approach to ensure we don

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-11 Thread via GitHub
lianetm commented on code in PR #16241: URL: https://github.com/apache/kafka/pull/16241#discussion_r1635211827 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -519,6 +524,7 @@ private void fetchOffsetsWithRetries(final Offse

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-11 Thread via GitHub
lianetm commented on code in PR #16241: URL: https://github.com/apache/kafka/pull/16241#discussion_r1635211827 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -519,6 +524,7 @@ private void fetchOffsetsWithRetries(final Offse

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-11 Thread via GitHub
lianetm commented on code in PR #16241: URL: https://github.com/apache/kafka/pull/16241#discussion_r1635192875 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -1279,4 +1288,119 @@ static class MemberInfo { this.m

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-11 Thread via GitHub
cadonna commented on code in PR #16241: URL: https://github.com/apache/kafka/pull/16241#discussion_r1635119225 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -1279,4 +1288,119 @@ static class MemberInfo { this.m

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-11 Thread via GitHub
lianetm commented on code in PR #16241: URL: https://github.com/apache/kafka/pull/16241#discussion_r1635060650 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -1279,4 +1288,119 @@ static class MemberInfo { this.m

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-11 Thread via GitHub
lianetm commented on code in PR #16241: URL: https://github.com/apache/kafka/pull/16241#discussion_r1635060650 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -1279,4 +1288,119 @@ static class MemberInfo { this.m

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-11 Thread via GitHub
lianetm commented on code in PR #16241: URL: https://github.com/apache/kafka/pull/16241#discussion_r1635060650 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -1279,4 +1288,119 @@ static class MemberInfo { this.m

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-11 Thread via GitHub
cadonna commented on code in PR #16241: URL: https://github.com/apache/kafka/pull/16241#discussion_r1634896716 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -1279,4 +1288,119 @@ static class MemberInfo { this.m

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-11 Thread via GitHub
AndrewJSchofield commented on code in PR #16241: URL: https://github.com/apache/kafka/pull/16241#discussion_r1634382890 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -1294,4 +1288,119 @@ static class MemberInfo {

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-10 Thread via GitHub
kirktrue commented on code in PR #16241: URL: https://github.com/apache/kafka/pull/16241#discussion_r1634066822 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -514,10 +514,11 @@ private void fetchOffsetsWithRetries(final Of

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-10 Thread via GitHub
kirktrue commented on code in PR #16241: URL: https://github.com/apache/kafka/pull/16241#discussion_r1634062217 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -1132,12 +1133,26 @@ private CompletableFuture> addOffsetFetch

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-10 Thread via GitHub
kirktrue commented on code in PR #16241: URL: https://github.com/apache/kafka/pull/16241#discussion_r1634061316 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -1132,12 +1133,26 @@ private CompletableFuture> addOffsetFetch

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-10 Thread via GitHub
cadonna commented on code in PR #16241: URL: https://github.com/apache/kafka/pull/16241#discussion_r1633243629 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -514,10 +514,11 @@ private void fetchOffsetsWithRetries(final Off

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-10 Thread via GitHub
cadonna commented on code in PR #16241: URL: https://github.com/apache/kafka/pull/16241#discussion_r1633243629 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -514,10 +514,11 @@ private void fetchOffsetsWithRetries(final Off

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-10 Thread via GitHub
cadonna commented on code in PR #16241: URL: https://github.com/apache/kafka/pull/16241#discussion_r1633243629 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -514,10 +514,11 @@ private void fetchOffsetsWithRetries(final Off

[PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-07 Thread via GitHub
kirktrue opened a new pull request, #16241: URL: https://github.com/apache/kafka/pull/16241 Don't remove the successful-but-expired offset fetch request from the inflight list until the next request. ### Committer Checklist (excluded from commit message) - [ ] Verify design and imp

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-06-03 Thread via GitHub
lianetm commented on PR #15844: URL: https://github.com/apache/kafka/pull/15844#issuecomment-2145677008 Hey @kirktrue , this is the simple integration test I was suggesting on the [other timeout PR](https://github.com/apache/kafka/pull/15640#issuecomment-2117763853), that should be an easy

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-05-31 Thread via GitHub
kirktrue closed pull request #15844: KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively URL: https://github.com/apache/kafka/pull/15844 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-05-16 Thread via GitHub
kirktrue commented on PR #15844: URL: https://github.com/apache/kafka/pull/15844#issuecomment-2116143157 Thanks for the review, @lianetm. Agreed on all your points. This PR is a draft because it's a PoC and more thought and tests are needed. -- This is an automated message from the Apache

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-05-16 Thread via GitHub
kirktrue commented on code in PR #15844: URL: https://github.com/apache/kafka/pull/15844#discussion_r1604004465 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -1145,14 +1141,42 @@ private CompletableFuture> addOffsetFetch

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-05-16 Thread via GitHub
lianetm commented on PR #15844: URL: https://github.com/apache/kafka/pull/15844#issuecomment-2115985840 FYI, I found another issue that looks to me it's related to this same situation: https://issues.apache.org/jira/browse/KAFKA-16777 -- This is an automated message from the Apache Git Se

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-05-13 Thread via GitHub
kirktrue commented on code in PR #15844: URL: https://github.com/apache/kafka/pull/15844#discussion_r1598899793 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -1145,14 +1141,42 @@ private CompletableFuture> addOffsetFetch

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-05-13 Thread via GitHub
kirktrue commented on code in PR #15844: URL: https://github.com/apache/kafka/pull/15844#discussion_r1598898939 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -1145,14 +1141,42 @@ private CompletableFuture> addOffsetFetch

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-05-06 Thread via GitHub
lianetm commented on PR #15844: URL: https://github.com/apache/kafka/pull/15844#issuecomment-2096232639 Did we consider the approach of simply decoupling the request timeout from the application event timeout? We could issue the fetch request without a time boundary (max value probably), an

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-05-06 Thread via GitHub
lianetm commented on PR #15844: URL: https://github.com/apache/kafka/pull/15844#issuecomment-2096155474 One concern on [comment](https://github.com/apache/kafka/pull/15844#discussion_r1591065112) above about how we identify this situation (inflight fetch requests that we shouldn't delete t

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-05-06 Thread via GitHub
lianetm commented on code in PR #15844: URL: https://github.com/apache/kafka/pull/15844#discussion_r1591065112 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -1145,14 +1141,42 @@ private CompletableFuture> addOffsetFetch

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-05-04 Thread via GitHub
AndrewJSchofield commented on code in PR #15844: URL: https://github.com/apache/kafka/pull/15844#discussion_r1589983081 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -1145,14 +1141,42 @@ private CompletableFuture> addOffset

Re: [PR] KAFKA-16637: AsyncKafkaConsumer removes offset fetch responses from cache too aggressively [kafka]

2024-05-02 Thread via GitHub
kirktrue commented on PR #15844: URL: https://github.com/apache/kafka/pull/15844#issuecomment-2091380043 > @kirktrue Should we rename the title of the PR/Jira now that we understand the root cause? Done šŸ‘ -- This is an automated message from the Apache Git Service. To respond to th