Re: [PR] KAFKA-16229: Fix slow expired producer id deletion [kafka]

2024-02-07 Thread via GitHub
jeqo commented on code in PR #15324: URL: https://github.com/apache/kafka/pull/15324#discussion_r1482430278 ## storage/src/main/java/org/apache/kafka/storage/internals/log/ProducerStateManager.java: ## @@ -177,7 +177,7 @@ private void addProducerId(long producerId, ProducerStat

[PR] KAFKA-16116 [kafka]

2024-02-07 Thread via GitHub
philipnee opened a new pull request, #15339: URL: https://github.com/apache/kafka/pull/15339 Adding the following rebalance metrics to the consumer: rebalance-latency-avg rebalance-latency-max rebalance-latency-total rebalance-rate-per-hour rebalance-total failed-rebalan

Re: [PR] Update subscriptions at the end. [kafka]

2024-02-07 Thread via GitHub
github-actions[bot] commented on PR #14720: URL: https://github.com/apache/kafka/pull/14720#issuecomment-1933309607 This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please ask a committer for review. If the PR has merge

Re: [PR] MINOR: Fix uses of ConfigException [kafka]

2024-02-07 Thread via GitHub
github-actions[bot] commented on PR #14721: URL: https://github.com/apache/kafka/pull/14721#issuecomment-1933309585 This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please ask a committer for review. If the PR has merge

Re: [PR] MINOR:Type Casting Correction AND Null Pointer Exception (NPE) Defense [kafka]

2024-02-07 Thread via GitHub
highluck commented on code in PR #9786: URL: https://github.com/apache/kafka/pull/9786#discussion_r1482300227 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/InternalStreamsBuilder.java: ## @@ -437,7 +438,10 @@ private void rewriteSingleStoreSelfJoin(

Re: [PR] KAFKA-15761: KRaft support in EpochDrivenReplicationProtocolAcceptanceTest [kafka]

2024-02-07 Thread via GitHub
highluck commented on PR #15295: URL: https://github.com/apache/kafka/pull/15295#issuecomment-1933212514 Oh thank you! I'll fix it right away -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] MINOR: remove un used code [kafka]

2024-02-07 Thread via GitHub
highluck commented on PR #15301: URL: https://github.com/apache/kafka/pull/15301#issuecomment-1933211994 @mimaison thank you! Can I please merge it? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[PR] Kafka streams scala3 [kafka]

2024-02-07 Thread via GitHub
mberndt123 opened a new pull request, #15338: URL: https://github.com/apache/kafka/pull/15338 A port of Kafka-Streams-Scala. The code itself was almost entirely compatible, most of the work revolves around the build system. Because core doesn't yet support Scala 3, it's necessa

Re: [PR] KAFKA-15462: Add Group Type Filter for List Group to the Admin Client [kafka]

2024-02-07 Thread via GitHub
rreddy-22 commented on PR #15150: URL: https://github.com/apache/kafka/pull/15150#issuecomment-1933178128 Ran all the tests locally with changes from this patch https://github.com/apache/kafka/pull/15311 and everything passes. Tests in ListConsumerGroupTest that use the new "consumer" proto

Re: [PR] KAFKA-15462: Add Group Type Filter for List Group to the Admin Client [kafka]

2024-02-07 Thread via GitHub
rreddy-22 commented on code in PR #15150: URL: https://github.com/apache/kafka/pull/15150#discussion_r1482258074 ## clients/src/test/java/org/apache/kafka/clients/admin/KafkaAdminClientTest.java: ## @@ -2804,8 +2804,8 @@ public void testListConsumerGroupsWithStates() throws Exc

Re: [PR] KAFKA-15462: Add Group Type Filter for List Group to the Admin Client [kafka]

2024-02-07 Thread via GitHub
rreddy-22 commented on code in PR #15150: URL: https://github.com/apache/kafka/pull/15150#discussion_r1482258074 ## clients/src/test/java/org/apache/kafka/clients/admin/KafkaAdminClientTest.java: ## @@ -2804,8 +2804,8 @@ public void testListConsumerGroupsWithStates() throws Exc

Re: [PR] KAFKA-15462: Add Group Type Filter for List Group to the Admin Client [kafka]

2024-02-07 Thread via GitHub
rreddy-22 commented on code in PR #15150: URL: https://github.com/apache/kafka/pull/15150#discussion_r1482157827 ## core/src/test/scala/unit/kafka/admin/ListConsumerGroupTest.scala: ## @@ -18,44 +18,47 @@ package kafka.admin import joptsimple.OptionException import org.junit

Re: [PR] KAFKA-15462: Add Group Type Filter for List Group to the Admin Client [kafka]

2024-02-07 Thread via GitHub
rreddy-22 commented on code in PR #15150: URL: https://github.com/apache/kafka/pull/15150#discussion_r1482168707 ## core/src/main/scala/kafka/admin/ConsumerGroupCommand.scala: ## @@ -187,16 +197,69 @@ object ConsumerGroupCommand extends Logging { } def listGroups():

Re: [PR] KAFKA-15462: Add Group Type Filter for List Group to the Admin Client [kafka]

2024-02-07 Thread via GitHub
rreddy-22 commented on code in PR #15150: URL: https://github.com/apache/kafka/pull/15150#discussion_r1482167679 ## core/src/main/scala/kafka/admin/ConsumerGroupCommand.scala: ## @@ -189,16 +199,68 @@ object ConsumerGroupCommand extends Logging { } def listGroups():

Re: [PR] KAFKA-15462: Add Group Type Filter for List Group to the Admin Client [kafka]

2024-02-07 Thread via GitHub
rreddy-22 commented on code in PR #15150: URL: https://github.com/apache/kafka/pull/15150#discussion_r1482157827 ## core/src/test/scala/unit/kafka/admin/ListConsumerGroupTest.scala: ## @@ -18,44 +18,47 @@ package kafka.admin import joptsimple.OptionException import org.junit

Re: [PR] KAFKA-16229: Fix slow expired producer id deletion [kafka]

2024-02-07 Thread via GitHub
jolshan commented on code in PR #15324: URL: https://github.com/apache/kafka/pull/15324#discussion_r1482140038 ## storage/src/main/java/org/apache/kafka/storage/internals/log/ProducerStateManager.java: ## @@ -177,7 +177,7 @@ private void addProducerId(long producerId, ProducerS

Re: [PR] KAFKA-16229: Fix slow expired producer id deletion [kafka]

2024-02-07 Thread via GitHub
jolshan commented on PR #15324: URL: https://github.com/apache/kafka/pull/15324#issuecomment-1932990160 Also @jeqo -- just curious which java version were you running? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [PR] KAFKA-15462: Add Group Type Filter for List Group to the Admin Client [kafka]

2024-02-07 Thread via GitHub
rreddy-22 commented on code in PR #15150: URL: https://github.com/apache/kafka/pull/15150#discussion_r1482112413 ## clients/src/main/java/org/apache/kafka/clients/admin/ConsumerGroupListing.java: ## @@ -21,14 +21,16 @@ import java.util.Optional; import org.apache.kafka.commo

Re: [PR] MINOR: Add MetadataType metric from KIP-866 #15299 [kafka]

2024-02-07 Thread via GitHub
mumrah commented on PR #15306: URL: https://github.com/apache/kafka/pull/15306#issuecomment-1932944727 @OmniaGM, the metrics changes were split out from my original PR into this one. The commit from my closed PR just has the batch size change https://github.com/apache/kafka/commit/12ce9c7f9

[jira] [Assigned] (KAFKA-16055) Thread unsafe use of HashMap stored in QueryableStoreProvider#storeProviders

2024-02-07 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman reassigned KAFKA-16055: -- Assignee: Kohei Nozaki (was: Kohei Nozaki) > Thread unsafe use of Ha

[jira] [Assigned] (KAFKA-16055) Thread unsafe use of HashMap stored in QueryableStoreProvider#storeProviders

2024-02-07 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman reassigned KAFKA-16055: -- Assignee: Kohei Nozaki > Thread unsafe use of HashMap stored in Query

[jira] [Updated] (KAFKA-16055) Thread unsafe use of HashMap stored in QueryableStoreProvider#storeProviders

2024-02-07 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman updated KAFKA-16055: --- Fix Version/s: 3.8.0 > Thread unsafe use of HashMap stored in QueryableStore

Re: [PR] KAFKA-16055: Thread unsafe use of HashMap stored in QueryableStoreProvider#storeProviders [kafka]

2024-02-07 Thread via GitHub
ableegoldman commented on PR #15121: URL: https://github.com/apache/kafka/pull/15121#issuecomment-1932890035 Merged to trunk. Thanks for the fix! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [PR] KAFKA-16055: Thread unsafe use of HashMap stored in QueryableStoreProvider#storeProviders [kafka]

2024-02-07 Thread via GitHub
ableegoldman merged PR #15121: URL: https://github.com/apache/kafka/pull/15121 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.ap

Re: [PR] MINOR: Fix some MetadataDelta handling issues during ZK migration [kafka]

2024-02-07 Thread via GitHub
cmccabe merged PR #15327: URL: https://github.com/apache/kafka/pull/15327 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[PR] Global state store restore custom processor [kafka]

2024-02-07 Thread via GitHub
wcarlson5 opened a new pull request, #15337: URL: https://github.com/apache/kafka/pull/15337 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of testi

[jira] [Assigned] (KAFKA-16001) Migrate ConsumerNetworkThreadTest away from ConsumerTestBuilder

2024-02-07 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True reassigned KAFKA-16001: - Assignee: (was: Kirk True) > Migrate ConsumerNetworkThreadTest away from ConsumerTestBuilde

[jira] [Updated] (KAFKA-15638) Investigate ConsumerNetworkThreadTest's testPollResultTimer

2024-02-07 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-15638: -- Labels: consumer-threading-refactor timeout unit-tests (was: consumer-threading-refactor unit-tests)

[jira] [Assigned] (KAFKA-15615) Improve handling of fetching during metadata updates

2024-02-07 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True reassigned KAFKA-15615: - Assignee: (was: Kirk True) > Improve handling of fetching during metadata updates > ---

[jira] [Assigned] (KAFKA-15639) Investigate ConsumerNetworkThreadTest's testResetPositionsProcessFailureIsIgnored

2024-02-07 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True reassigned KAFKA-15639: - Assignee: (was: Kirk True) > Investigate ConsumerNetworkThreadTest's > testResetPositionsP

[jira] [Commented] (KAFKA-14747) FK join should record discarded subscription responses

2024-02-07 Thread Ayoub Omari (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815424#comment-17815424 ] Ayoub Omari commented on KAFKA-14747: - I see that the ticket is open for a while. Ma

[jira] [Assigned] (KAFKA-16156) System test failing for new consumer on endOffsets with negative timestamps

2024-02-07 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True reassigned KAFKA-16156: - Assignee: Philip Nee > System test failing for new consumer on endOffsets with negative timesta

[jira] [Assigned] (KAFKA-16178) AsyncKafkaConsumer doesn't retry joining the group after rediscovering group coordinator

2024-02-07 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True reassigned KAFKA-16178: - Assignee: Lianet Magrans (was: Philip Nee) > AsyncKafkaConsumer doesn't retry joining the grou

[jira] [Updated] (KAFKA-15539) Client should stop fetching while partitions being revoked

2024-02-07 Thread Stanislav Kozlovski (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanislav Kozlovski updated KAFKA-15539: Fix Version/s: (was: 3.7.0) > Client should stop fetching while partitions bei

[jira] [Updated] (KAFKA-15631) Do not send new heartbeat request while another one in-flight

2024-02-07 Thread Stanislav Kozlovski (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanislav Kozlovski updated KAFKA-15631: Fix Version/s: (was: 3.7.0) > Do not send new heartbeat request while another

[jira] [Updated] (KAFKA-15540) Handle heartbeat and revocation when consumer leaves group

2024-02-07 Thread Stanislav Kozlovski (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanislav Kozlovski updated KAFKA-15540: Fix Version/s: (was: 3.7.0) > Handle heartbeat and revocation when consumer le

[jira] [Updated] (KAFKA-15543) Send HB request right after reconciliation completes

2024-02-07 Thread Stanislav Kozlovski (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanislav Kozlovski updated KAFKA-15543: Fix Version/s: (was: 3.7.0) > Send HB request right after reconciliation compl

[jira] [Updated] (KAFKA-15573) Implement auto-commit on partition assignment revocation

2024-02-07 Thread Stanislav Kozlovski (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanislav Kozlovski updated KAFKA-15573: Fix Version/s: (was: 3.7.0) > Implement auto-commit on partition assignment re

[jira] [Updated] (KAFKA-15592) Member does not need to always try to join a group when a groupId is configured

2024-02-07 Thread Stanislav Kozlovski (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanislav Kozlovski updated KAFKA-15592: Fix Version/s: (was: 3.7.0) > Member does not need to always try to join a gro

[jira] [Updated] (KAFKA-15515) Remove duplicated integration tests for new consumer

2024-02-07 Thread Stanislav Kozlovski (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanislav Kozlovski updated KAFKA-15515: Fix Version/s: (was: 3.7.0) > Remove duplicated integration tests for new cons

[jira] [Updated] (KAFKA-15651) Investigate auto commit guarantees during Consumer.assign()

2024-02-07 Thread Stanislav Kozlovski (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanislav Kozlovski updated KAFKA-15651: Fix Version/s: (was: 3.7.0) > Investigate auto commit guarantees during Consum

[jira] [Updated] (KAFKA-15694) New integration tests to have full coverage for preview

2024-02-07 Thread Stanislav Kozlovski (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanislav Kozlovski updated KAFKA-15694: Fix Version/s: (was: 3.7.0) > New integration tests to have full coverage for

[jira] [Updated] (KAFKA-15840) Correct initialization of ConsumerGroupHeartbeat by client

2024-02-07 Thread Stanislav Kozlovski (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanislav Kozlovski updated KAFKA-15840: Fix Version/s: (was: 3.7.0) > Correct initialization of ConsumerGroupHeartbeat

[jira] [Updated] (KAFKA-15694) New integration tests to have full coverage for preview

2024-02-07 Thread Stanislav Kozlovski (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanislav Kozlovski updated KAFKA-15694: Description: These are to fix bugs discovered during PR reviews but not tests. ย 

[jira] [Updated] (KAFKA-16121) Partition reassignments in ZK migration dual write mode stalled until leader epoch incremented

2024-02-07 Thread Stanislav Kozlovski (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanislav Kozlovski updated KAFKA-16121: Fix Version/s: (was: 3.7.0) > Partition reassignments in ZK migration dual wri

[jira] [Closed] (KAFKA-16121) Partition reassignments in ZK migration dual write mode stalled until leader epoch incremented

2024-02-07 Thread Stanislav Kozlovski (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanislav Kozlovski closed KAFKA-16121. --- > Partition reassignments in ZK migration dual write mode stalled until leader > epoch

[jira] [Closed] (KAFKA-15539) Client should stop fetching while partitions being revoked

2024-02-07 Thread Stanislav Kozlovski (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanislav Kozlovski closed KAFKA-15539. --- > Client should stop fetching while partitions being revoked > -

[jira] [Closed] (KAFKA-15515) Remove duplicated integration tests for new consumer

2024-02-07 Thread Stanislav Kozlovski (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanislav Kozlovski closed KAFKA-15515. --- > Remove duplicated integration tests for new consumer > ---

[jira] [Closed] (KAFKA-15543) Send HB request right after reconciliation completes

2024-02-07 Thread Stanislav Kozlovski (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanislav Kozlovski closed KAFKA-15543. --- > Send HB request right after reconciliation completes > ---

[jira] [Closed] (KAFKA-15631) Do not send new heartbeat request while another one in-flight

2024-02-07 Thread Stanislav Kozlovski (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanislav Kozlovski closed KAFKA-15631. --- > Do not send new heartbeat request while another one in-flight > --

[jira] [Closed] (KAFKA-15573) Implement auto-commit on partition assignment revocation

2024-02-07 Thread Stanislav Kozlovski (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanislav Kozlovski closed KAFKA-15573. --- > Implement auto-commit on partition assignment revocation > ---

[jira] [Closed] (KAFKA-15592) Member does not need to always try to join a group when a groupId is configured

2024-02-07 Thread Stanislav Kozlovski (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanislav Kozlovski closed KAFKA-15592. --- > Member does not need to always try to join a group when a groupId is > configured > -

[jira] [Commented] (KAFKA-16121) Partition reassignments in ZK migration dual write mode stalled until leader epoch incremented

2024-02-07 Thread Stanislav Kozlovski (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815421#comment-17815421 ] Stanislav Kozlovski commented on KAFKA-16121: - Marked as closed in order to

[jira] [Closed] (KAFKA-15840) Correct initialization of ConsumerGroupHeartbeat by client

2024-02-07 Thread Stanislav Kozlovski (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanislav Kozlovski closed KAFKA-15840. --- > Correct initialization of ConsumerGroupHeartbeat by client > -

[jira] [Closed] (KAFKA-16121) Partition reassignments in ZK migration dual write mode stalled until leader epoch incremented

2024-02-07 Thread Stanislav Kozlovski (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanislav Kozlovski closed KAFKA-16121. --- > Partition reassignments in ZK migration dual write mode stalled until leader > epoch

Re: [PR] KAFKA-15462: Add Group Type Filter for List Group to the Admin Client [kafka]

2024-02-07 Thread via GitHub
rreddy-22 commented on code in PR #15150: URL: https://github.com/apache/kafka/pull/15150#discussion_r1482012622 ## clients/src/main/java/org/apache/kafka/clients/admin/ConsumerGroupListing.java: ## @@ -68,48 +97,44 @@ public boolean isSimpleConsumerGroup() { } /** -

Re: [PR] KAFKA-15462: Add Group Type Filter for List Group to the Admin Client [kafka]

2024-02-07 Thread via GitHub
rreddy-22 commented on code in PR #15150: URL: https://github.com/apache/kafka/pull/15150#discussion_r1481994449 ## core/src/test/scala/unit/kafka/admin/ListConsumerGroupTest.scala: ## @@ -64,28 +64,89 @@ class ListConsumerGroupTest extends ConsumerGroupCommandTest { val s

Re: [PR] KAFKA-14683 Migrate WorkerSinkTaskTest to Mockito (2/3) [kafka]

2024-02-07 Thread via GitHub
gharris1727 commented on code in PR #15313: URL: https://github.com/apache/kafka/pull/15313#discussion_r1481920004 ## connect/runtime/src/test/java/org/apache/kafka/connect/runtime/WorkerSinkTaskMockitoTest.java: ## @@ -343,6 +354,103 @@ public void testShutdown() throws Excepti

Re: [PR] MINOR Fix a case where not all ACLs for a given resource are written to ZK [kafka]

2024-02-07 Thread via GitHub
cmccabe commented on PR #15327: URL: https://github.com/apache/kafka/pull/15327#issuecomment-193270 Thanks, @mumrah . It looks good. One comment: it seems like any error log being issued should fail any junit test, unless itโ€™s expected, right? -- This is an automated message from the

[jira] [Commented] (KAFKA-16235) auto commit still causes delays due to retriable UNKNOWN_TOPIC_OR_PARTITION

2024-02-07 Thread Ryan Leslie (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815407#comment-17815407 ] Ryan Leslie commented on KAFKA-16235: - Linked related JIRAs. > auto commit still ca

[jira] [Created] (KAFKA-16235) auto commit still causes delays due to retriable UNKNOWN_TOPIC_OR_PARTITION

2024-02-07 Thread Ryan Leslie (Jira)
Ryan Leslie created KAFKA-16235: --- Summary: auto commit still causes delays due to retriable UNKNOWN_TOPIC_OR_PARTITION Key: KAFKA-16235 URL: https://issues.apache.org/jira/browse/KAFKA-16235 Project: Ka

[jira] [Updated] (KAFKA-16233) Review auto-commit continuously committing when no progress

2024-02-07 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-16233: -- Labels: consumer-threading-refactor (was: ) > Review auto-commit continuously committing when no prog

[jira] [Updated] (KAFKA-14920) Address timeouts and out of order sequences

2024-02-07 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan updated KAFKA-14920: --- Description: KAFKA-14844 showed the destructive nature of a timeout on the first produce re

[jira] [Commented] (KAFKA-13292) InvalidPidMappingException: The producer attempted to use a producer id which is not currently assigned to its transactional id

2024-02-07 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815404#comment-17815404 ] Matthias J. Sax commented on KAFKA-13292: - Sounds like a question about Spring..

[jira] [Commented] (KAFKA-16234) Log directory failure re-creates partitions in another logdir automatically

2024-02-07 Thread Gaurav Narula (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815403#comment-17815403 ] Gaurav Narula commented on KAFKA-16234: --- Perhaps a way to solve this would be to d

Re: [PR] KAFKA-16226 Reduce synchronization between producer threads [kafka]

2024-02-07 Thread via GitHub
hachikuji commented on code in PR #15323: URL: https://github.com/apache/kafka/pull/15323#discussion_r1481936087 ## clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java: ## @@ -647,27 +647,27 @@ private long batchReady(boolean exhausted, Topic

[jira] [Commented] (KAFKA-16217) Transactional producer stuck in IllegalStateException during close

2024-02-07 Thread Calvin Liu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815402#comment-17815402 ] Calvin Liu commented on KAFKA-16217: [~kirktrue] I have a UT which simulate the clos

[PR] [No review] Repro kafka-16217 [kafka]

2024-02-07 Thread via GitHub
CalvinConfluent opened a new pull request, #15336: URL: https://github.com/apache/kafka/pull/15336 A UT to repro the bug in https://issues.apache.org/jira/browse/KAFKA-16217 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[jira] [Commented] (KAFKA-16234) Log directory failure re-creates partitions in another logdir automatically

2024-02-07 Thread Gaurav Narula (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815397#comment-17815397 ] Gaurav Narula commented on KAFKA-16234: --- This gets trickier because {{LogManager::

Re: [PR] KAFKA-14822: Allow restricting File and Directory ConfigProviders to specific paths [kafka]

2024-02-07 Thread via GitHub
gharris1727 commented on code in PR #14995: URL: https://github.com/apache/kafka/pull/14995#discussion_r1481872234 ## clients/src/test/java/org/apache/kafka/common/config/provider/MockFileConfigProvider.java: ## @@ -42,10 +43,12 @@ public void configure(Map configs) { }

[PR] KAFKA-16234: Add topicId to Partition constructor [kafka]

2024-02-07 Thread via GitHub
OmniaGM opened a new pull request, #15335: URL: https://github.com/apache/kafka/pull/15335 This pr fixes the bug created by #15263 which caused topic partition to be recreated whenever the original log dir is offline. I believe the bug #15263 was trying to fix is more rare to happened th

[jira] [Commented] (KAFKA-15467) Kafka broker returns offset out of range for topic/partitions on restart from unclean shutdown

2024-02-07 Thread Steve Jacobs (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815385#comment-17815385 ] Steve Jacobs commented on KAFKA-15467: -- The way to reproduce this is an unclean shu

Re: [PR] KAFKA-16226 Reduce synchronization between producer threads [kafka]

2024-02-07 Thread via GitHub
hachikuji commented on code in PR #15323: URL: https://github.com/apache/kafka/pull/15323#discussion_r1481854476 ## clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java: ## @@ -647,27 +647,27 @@ private long batchReady(boolean exhausted, Topic

Re: [PR] streams-scala: remove collections-compat dependency when on Scala 2.13 [kafka]

2024-02-07 Thread via GitHub
mberndt123 commented on PR #15239: URL: https://github.com/apache/kafka/pull/15239#issuecomment-1932541985 > Do we want to remove it from streams only or also for core? I've tried that and thought it worked because of a silly mistake that I made. But `core` actually needs it, so it ne

Re: [PR] KAFKA-15670: add "inter.broker.listener.name" config in KRaft controller config [kafka]

2024-02-07 Thread via GitHub
mimaison commented on code in PR #14631: URL: https://github.com/apache/kafka/pull/14631#discussion_r1481826420 ## docs/ops.html: ## @@ -3819,6 +3819,12 @@ Provisioning the KRaft controller quorum # ZooKeeper client configuration zookeeper.connect=localhost:2181 +# The inter

Re: [PR] mention IdentityReplicationPolicy in ops docs [kafka]

2024-02-07 Thread via GitHub
mimaison commented on PR #10983: URL: https://github.com/apache/kafka/pull/10983#issuecomment-1932493933 @showuon I pushed a commit to tweak this section. Can you take another look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

Re: [PR] KAFKA-16154: Broker returns offset for LATEST_TIERED_TIMESTAMP [kafka]

2024-02-07 Thread via GitHub
satishd commented on code in PR #15213: URL: https://github.com/apache/kafka/pull/15213#discussion_r1481778242 ## core/src/main/scala/kafka/log/UnifiedLog.scala: ## @@ -1300,18 +1301,29 @@ class UnifiedLog(@volatile var logStartOffset: Long, } else if (targetTimestamp ==

Re: [PR] KAFKA-16229: Fix slow expired producer id deletion [kafka]

2024-02-07 Thread via GitHub
jeqo commented on PR #15324: URL: https://github.com/apache/kafka/pull/15324#issuecomment-1932429115 @jolshan sure! I just added it ๐Ÿ‘๐Ÿฝ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] KAFKA-16226 Reduce synchronization between producer threads [kafka]

2024-02-07 Thread via GitHub
msn-tldr commented on PR #15323: URL: https://github.com/apache/kafka/pull/15323#issuecomment-1932366816 @hachikuji There are unrelated test failures on Jenkins run. Further looking at history of failed tests, they have been failing from before. https://ge.apache.org/s/fr7yermmdio

[jira] [Updated] (KAFKA-16234) Log directory failure re-creates partitions in another logdir automatically

2024-02-07 Thread Gaurav Narula (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gaurav Narula updated KAFKA-16234: -- Description: With [KAFKA-16157|https://github.com/apache/kafka/pull/15263] we made changes in

[jira] [Assigned] (KAFKA-16234) Log directory failure re-creates partitions in another logdir automatically

2024-02-07 Thread Gaurav Narula (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gaurav Narula reassigned KAFKA-16234: - Assignee: Omnia Ibrahim > Log directory failure re-creates partitions in another logdir

[jira] [Updated] (KAFKA-16234) Log directory failure re-creates partitions in another logdir automatically

2024-02-07 Thread Gaurav Narula (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gaurav Narula updated KAFKA-16234: -- Description: With [KAFKA-16157|https://github.com/apache/kafka/pull/15263] we made changes in

[jira] [Created] (KAFKA-16234) Log directory failure re-creates partitions in another logdir automatically

2024-02-07 Thread Gaurav Narula (Jira)
Gaurav Narula created KAFKA-16234: - Summary: Log directory failure re-creates partitions in another logdir automatically Key: KAFKA-16234 URL: https://issues.apache.org/jira/browse/KAFKA-16234 Project

Re: [PR] KAFKA-7632: Support Compression Level [kafka]

2024-02-07 Thread via GitHub
ijuma commented on code in PR #10826: URL: https://github.com/apache/kafka/pull/10826#discussion_r850506417 ## clients/src/main/java/org/apache/kafka/clients/producer/ProducerConfig.java: ## @@ -188,6 +190,12 @@ public class ProducerConfig extends AbstractConfig {

Re: [PR] KAFKA-14133: Move consumer mock in TaskManagerTest to Mockito - part 2 [kafka]

2024-02-07 Thread via GitHub
clolov commented on PR #15261: URL: https://github.com/apache/kafka/pull/15261#issuecomment-1932183121 Heya @cadonna, apologies for the delay. I am not certain I fully understand the comments, so I wanted to confirm before making changes. The purpose of this pull request is to just migrate

Re: [PR] KAFKA-14133: Move consumer mock in TaskManagerTest to Mockito - part 2 [kafka]

2024-02-07 Thread via GitHub
clolov commented on code in PR #15261: URL: https://github.com/apache/kafka/pull/15261#discussion_r1481566534 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/TaskManagerTest.java: ## @@ -2016,13 +2015,14 @@ public void shouldReleaseLockForUnassignedTasksAf

Re: [PR] KAFKA-14133: Move consumer mock in TaskManagerTest to Mockito - part 2 [kafka]

2024-02-07 Thread via GitHub
clolov commented on code in PR #15261: URL: https://github.com/apache/kafka/pull/15261#discussion_r1481561763 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/TaskManagerTest.java: ## @@ -2399,7 +2393,8 @@ public void shouldCloseActiveTasksWhenHandlingLostTa

Re: [PR] KAFKA-14133: Move consumer mock in TaskManagerTest to Mockito - part 2 [kafka]

2024-02-07 Thread via GitHub
clolov commented on code in PR #15261: URL: https://github.com/apache/kafka/pull/15261#discussion_r1481561392 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/TaskManagerTest.java: ## @@ -2332,19 +2335,12 @@ public void shouldCloseActiveUnassignedSuspendedT

Re: [PR] KAFKA-14133: Move consumer mock in TaskManagerTest to Mockito - part 2 [kafka]

2024-02-07 Thread via GitHub
clolov commented on code in PR #15261: URL: https://github.com/apache/kafka/pull/15261#discussion_r1481560896 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/TaskManagerTest.java: ## @@ -2210,6 +2210,9 @@ public void shouldComputeOffsetSumForStandbyTask() t

Re: [PR] KAFKA-14133: Move consumer mock in TaskManagerTest to Mockito - part 2 [kafka]

2024-02-07 Thread via GitHub
clolov commented on code in PR #15261: URL: https://github.com/apache/kafka/pull/15261#discussion_r1481557433 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/TaskManagerTest.java: ## @@ -2369,12 +2366,9 @@ public void closeClean() { }

Re: [PR] KAFKA-14133: Move consumer mock in TaskManagerTest to Mockito - part 2 [kafka]

2024-02-07 Thread via GitHub
clolov commented on code in PR #15261: URL: https://github.com/apache/kafka/pull/15261#discussion_r1481556951 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/TaskManagerTest.java: ## @@ -2332,19 +2335,12 @@ public void shouldCloseActiveUnassignedSuspendedT

Re: [PR] KAFKA-14133: Move consumer mock in TaskManagerTest to Mockito - part 2 [kafka]

2024-02-07 Thread via GitHub
clolov commented on code in PR #15261: URL: https://github.com/apache/kafka/pull/15261#discussion_r1481555805 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/TaskManagerTest.java: ## @@ -4821,8 +4911,10 @@ private Map handleAssignment(final Map assignment

[jira] [Updated] (KAFKA-16224) Fix handling of deleted topic when auto-committing before revocation

2024-02-07 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans updated KAFKA-16224: --- Description: Current logic for auto-committing offsets when partitions are revoked will ret

Re: [PR] KAFKA-16154: Broker returns offset for LATEST_TIERED_TIMESTAMP [kafka]

2024-02-07 Thread via GitHub
clolov commented on PR #15213: URL: https://github.com/apache/kafka/pull/15213#issuecomment-1932149971 Heya @showuon @kamalcph @satishd, I hope I have addressed the latest comments! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

[jira] [Created] (KAFKA-16233) Review auto-commit continuously committing when no progress

2024-02-07 Thread Lianet Magrans (Jira)
Lianet Magrans created KAFKA-16233: -- Summary: Review auto-commit continuously committing when no progress Key: KAFKA-16233 URL: https://issues.apache.org/jira/browse/KAFKA-16233 Project: Kafka

Re: [PR] KAFKA-16154: Broker returns offset for LATEST_TIERED_TIMESTAMP [kafka]

2024-02-07 Thread via GitHub
clolov commented on code in PR #15213: URL: https://github.com/apache/kafka/pull/15213#discussion_r1481540437 ## core/src/main/scala/kafka/log/UnifiedLog.scala: ## @@ -1300,18 +1301,29 @@ class UnifiedLog(@volatile var logStartOffset: Long, } else if (targetTimestamp ==

Re: [PR] KAFKA-16226 Reduce synchronization between producer threads [kafka]

2024-02-07 Thread via GitHub
msn-tldr commented on code in PR #15323: URL: https://github.com/apache/kafka/pull/15323#discussion_r1481455003 ## clients/src/main/java/org/apache/kafka/clients/producer/internals/RecordAccumulator.java: ## @@ -647,27 +647,27 @@ private long batchReady(boolean exhausted, TopicP

[jira] [Updated] (KAFKA-16226) Java client: Performance regression in Trogdor benchmark with high partition counts

2024-02-07 Thread Mayank Shekhar Narula (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mayank Shekhar Narula updated KAFKA-16226: -- Description: h1. Background https://issues.apache.org/jira/browse/KAFKA-15415

[jira] [Updated] (KAFKA-15824) SubscriptionState's maybeValidatePositionForCurrentLeader should handle partition which isn't subscribed yet

2024-02-07 Thread Mayank Shekhar Narula (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mayank Shekhar Narula updated KAFKA-15824: -- Description: As can be [maybeValidatePositionForCurrentLeader|https://github.

[jira] [Updated] (KAFKA-15824) SubscriptionState's maybeValidatePositionForCurrentLeader should handle partition which isn't subscribed yet

2024-02-07 Thread Mayank Shekhar Narula (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mayank Shekhar Narula updated KAFKA-15824: -- Description: As can be [maybeValidatePositionForCurrentLeader|https://github.

  1   2   >