Re: [PR] KAFKA-16654:Refactor kafka.test.annotation.Type and ClusterTestExtensions [kafka]

2024-05-17 Thread via GitHub
TaiJuWu commented on code in PR #15916: URL: https://github.com/apache/kafka/pull/15916#discussion_r1605682673 ## core/src/test/java/kafka/test/junit/ClusterTestExtensions.java: ## @@ -122,31 +115,57 @@ public Stream provideTestTemplateInvocationContex return generated

Re: [PR] KAFKA-16992: InvalidRequestException: ADD_PARTITIONS_TO_TXN with version 4 which is not enabled when upgrading from kafka [kafka]

2024-05-17 Thread via GitHub
jolshan merged PR #15971: URL: https://github.com/apache/kafka/pull/15971 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[jira] [Commented] (KAFKA-16791) Add thread detection to ClusterTestExtensions

2024-05-17 Thread bboyleonp (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847482#comment-17847482 ] bboyleonp commented on KAFKA-16791: --- Hi [~chia7712], I am interested in this one. May

Re: [PR] KAFKA-15045: (KIP-924 pt. 5) Add rack information to ApplicationState [kafka]

2024-05-17 Thread via GitHub
ableegoldman commented on code in PR #15972: URL: https://github.com/apache/kafka/pull/15972#discussion_r1605660608 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamsPartitionAssignor.java: ## @@ -472,23 +482,74 @@ public GroupAssignment assign(final C

Re: [PR] KAFKA-15045: (KIP-924 pt. 5) Add rack information to ApplicationState [kafka]

2024-05-17 Thread via GitHub
ableegoldman commented on code in PR #15972: URL: https://github.com/apache/kafka/pull/15972#discussion_r1605659907 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamsPartitionAssignor.java: ## @@ -472,23 +482,74 @@ public GroupAssignment assign(final C

Re: [PR] KAFKA-15045: (KIP-924 pt. 5) Add rack information to ApplicationState [kafka]

2024-05-17 Thread via GitHub
ableegoldman commented on code in PR #15972: URL: https://github.com/apache/kafka/pull/15972#discussion_r1605659120 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamsPartitionAssignor.java: ## @@ -472,23 +482,74 @@ public GroupAssignment assign(final C

Re: [PR] KAFKA-15045: (KIP-924 pt. 5) Add rack information to ApplicationState [kafka]

2024-05-17 Thread via GitHub
ableegoldman commented on code in PR #15972: URL: https://github.com/apache/kafka/pull/15972#discussion_r1605656530 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamsPartitionAssignor.java: ## @@ -472,23 +482,74 @@ public GroupAssignment assign(final C

Re: [PR] KAFKA-15045: (KIP-924 pt. 5) Add rack information to ApplicationState [kafka]

2024-05-17 Thread via GitHub
ableegoldman commented on code in PR #15972: URL: https://github.com/apache/kafka/pull/15972#discussion_r1605639072 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamsPartitionAssignor.java: ## @@ -472,23 +482,74 @@ public GroupAssignment assign(final C

Re: [PR] KAFKA-15974: Enforce that event processing respects user-provided timeout [kafka]

2024-05-17 Thread via GitHub
kirktrue commented on PR #15640: URL: https://github.com/apache/kafka/pull/15640#issuecomment-2118579935 @lianetm @cadonna—I believe I have addressed all the actionable feedback. Are there additional concerns about this PR that prevent it from being merged? Thanks. -- This is an automate

Re: [PR] KAFKA-15974: Enforce that event processing respects user-provided timeout [kafka]

2024-05-17 Thread via GitHub
kirktrue commented on PR #15640: URL: https://github.com/apache/kafka/pull/15640#issuecomment-2118578191 > Hey @cadonna, the tricky bit is that, for some events, the request managers do expire requests too, so in this flow you described: > > > The event is processed in the Application

Re: [PR] MINOR: fix flaky testRecordThreadIdleRatio [kafka]

2024-05-17 Thread via GitHub
gaurav-narula commented on code in PR #15987: URL: https://github.com/apache/kafka/pull/15987#discussion_r1605528394 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/runtime/MultiThreadedEventProcessorTest.java: ## @@ -61,8 +61,11 @@ public DelayEventAccumul

Re: [PR] KAFKA-16625: Reverse lookup map from topic partitions to members [kafka]

2024-05-17 Thread via GitHub
jeffkbkim commented on code in PR #15974: URL: https://github.com/apache/kafka/pull/15974#discussion_r1605535757 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/OptimizedUniformAssignmentBuilder.java: ## @@ -59,7 +59,7 @@ public class OptimizedUnif

Re: [PR] MINOR: fix flaky testRecordThreadIdleRatio [kafka]

2024-05-17 Thread via GitHub
gaurav-narula commented on code in PR #15987: URL: https://github.com/apache/kafka/pull/15987#discussion_r1605530429 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/runtime/MultiThreadedEventProcessorTest.java: ## @@ -61,8 +61,11 @@ public DelayEventAccumul

Re: [PR] MINOR: fix flaky testRecordThreadIdleRatio [kafka]

2024-05-17 Thread via GitHub
gaurav-narula commented on code in PR #15987: URL: https://github.com/apache/kafka/pull/15987#discussion_r1605530429 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/runtime/MultiThreadedEventProcessorTest.java: ## @@ -61,8 +61,11 @@ public DelayEventAccumul

Re: [PR] MINOR: fix flaky testRecordThreadIdleRatio [kafka]

2024-05-17 Thread via GitHub
gaurav-narula commented on code in PR #15987: URL: https://github.com/apache/kafka/pull/15987#discussion_r1605528394 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/runtime/MultiThreadedEventProcessorTest.java: ## @@ -61,8 +61,11 @@ public DelayEventAccumul

Re: [PR] MINOR: fix flaky testRecordThreadIdleRatio [kafka]

2024-05-17 Thread via GitHub
gaurav-narula commented on code in PR #15987: URL: https://github.com/apache/kafka/pull/15987#discussion_r1605525333 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/runtime/MultiThreadedEventProcessorTest.java: ## @@ -61,8 +61,11 @@ public DelayEventAccumul

Re: [PR] MINOR: fix flaky testRecordThreadIdleRatio [kafka]

2024-05-17 Thread via GitHub
jeffkbkim commented on code in PR #15987: URL: https://github.com/apache/kafka/pull/15987#discussion_r1605519141 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/runtime/MultiThreadedEventProcessorTest.java: ## @@ -61,8 +61,11 @@ public DelayEventAccumulator

[PR] KAFKA-16793: Heartbeat API for upgrading ConsumerGroup [kafka]

2024-05-17 Thread via GitHub
dongnuo123 opened a new pull request, #15988: URL: https://github.com/apache/kafka/pull/15988 This patch implements the heartbeat api to the members that use the classic protocol in a ConsumerGroup. ### Committer Checklist (excluded from commit message) - [ ] Verify design and impl

[jira] [Created] (KAFKA-16793) Heartbeat API for upgrading ConsumerGroup

2024-05-17 Thread Dongnuo Lyu (Jira)
Dongnuo Lyu created KAFKA-16793: --- Summary: Heartbeat API for upgrading ConsumerGroup Key: KAFKA-16793 URL: https://issues.apache.org/jira/browse/KAFKA-16793 Project: Kafka Issue Type: Sub-task

Re: [PR] MINOR: Remove deprecated constructors from Connect's Kafka*BackingStore classes [kafka]

2024-05-17 Thread via GitHub
chia7712 merged PR #15865: URL: https://github.com/apache/kafka/pull/15865 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

Re: [PR] MINOR: fix flaky testRecordThreadIdleRatio [kafka]

2024-05-17 Thread via GitHub
gaurav-narula commented on code in PR #15987: URL: https://github.com/apache/kafka/pull/15987#discussion_r1605472276 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/runtime/MultiThreadedEventProcessorTest.java: ## @@ -475,9 +478,7 @@ public void testRecordT

Re: [PR] KAFKA-16223: Replace EasyMock/PowerMock with Mockito for KafkaConfigBackingStoreTest [kafka]

2024-05-17 Thread via GitHub
chia7712 merged PR #15933: URL: https://github.com/apache/kafka/pull/15933 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[jira] [Resolved] (KAFKA-15723) KRaft support in ListOffsetsRequestTest

2024-05-17 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-15723. Fix Version/s: 3.8.0 Resolution: Fixed > KRaft support in ListOffsetsRequestTest >

Re: [PR] KAFKA-15723: KRaft support in ListOffsetsRequestTest [kafka]

2024-05-17 Thread via GitHub
chia7712 merged PR #15980: URL: https://github.com/apache/kafka/pull/15980 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[jira] [Resolved] (KAFKA-16544) DescribeTopicsResult#allTopicIds and DescribeTopicsResult#allTopicNames should return null instead of throwing NPE

2024-05-17 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-16544. Fix Version/s: 3.8 Resolution: Fixed > DescribeTopicsResult#allTopicIds and Describ

Re: [PR] KAFKA-16544 DescribeTopicsResult#allTopicIds and DescribeTopicsResult#allTopicNames should return null instead of throwing NPE [kafka]

2024-05-17 Thread via GitHub
chia7712 merged PR #15979: URL: https://github.com/apache/kafka/pull/15979 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

Re: [PR] KAFKA-7342: Migrate tests in o.a.k.streams to JUnit 5 (except KafkaStreamsTest) [kafka]

2024-05-17 Thread via GitHub
chia7712 merged PR #15942: URL: https://github.com/apache/kafka/pull/15942 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[jira] [Updated] (KAFKA-16792) Enable consumer unit tests that fail to fetch offsets only for new consumer with poll(0)

2024-05-17 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans updated KAFKA-16792: --- Summary: Enable consumer unit tests that fail to fetch offsets only for new consumer with po

[jira] [Assigned] (KAFKA-16764) New consumer should throw InvalidTopicException on poll when invalid topic in metadata

2024-05-17 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans reassigned KAFKA-16764: -- Assignee: appchemist > New consumer should throw InvalidTopicException on poll when i

[jira] [Commented] (KAFKA-16764) New consumer should throw InvalidTopicException on poll when invalid topic in metadata

2024-05-17 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847427#comment-17847427 ] Lianet Magrans commented on KAFKA-16764: Sure! Thanks for helping out! Follow-up

[jira] [Created] (KAFKA-16792) Enable related unit tests that fail only for new consumer with to poll(0)

2024-05-17 Thread Lianet Magrans (Jira)
Lianet Magrans created KAFKA-16792: -- Summary: Enable related unit tests that fail only for new consumer with to poll(0) Key: KAFKA-16792 URL: https://issues.apache.org/jira/browse/KAFKA-16792 Project

[jira] [Updated] (KAFKA-16792) Enable related unit tests that fail to fetch offsets only for new consumer with to poll(0)

2024-05-17 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans updated KAFKA-16792: --- Summary: Enable related unit tests that fail to fetch offsets only for new consumer with to

[jira] [Updated] (KAFKA-16792) Enable related unit tests that fail to fetch offsets only for new consumer with to poll(0)

2024-05-17 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans updated KAFKA-16792: --- Description: Enable the following unit tests for the new async consumer in KafkaConsumerTest

[jira] [Created] (KAFKA-16791) Add thread detection to ClusterTestExtensions

2024-05-17 Thread Chia-Ping Tsai (Jira)
Chia-Ping Tsai created KAFKA-16791: -- Summary: Add thread detection to ClusterTestExtensions Key: KAFKA-16791 URL: https://issues.apache.org/jira/browse/KAFKA-16791 Project: Kafka Issue Type:

[PR] MINOR: fix flaky testRecordThreadIdleRatio [kafka]

2024-05-17 Thread via GitHub
jeffkbkim opened a new pull request, #15987: URL: https://github.com/apache/kafka/pull/15987 DelayEventAccumulator should return immediately if there are no events in the queue. Also removed some unused fields inside EventProcessorThread ### Committer Checklist (excluded from commit m

Re: [PR] MINOR: use try-with resources in ClientMetricsManagerTest [kafka]

2024-05-17 Thread via GitHub
chia7712 merged PR #15982: URL: https://github.com/apache/kafka/pull/15982 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

Re: [PR] MINOR: Add debug enablement check when using log.debug [kafka]

2024-05-17 Thread via GitHub
chia7712 merged PR #15977: URL: https://github.com/apache/kafka/pull/15977 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

Re: [PR] KAFKA-16789: Fix thread leak detection for event handler threads [kafka]

2024-05-17 Thread via GitHub
chia7712 commented on code in PR #15984: URL: https://github.com/apache/kafka/pull/15984#discussion_r1605427011 ## core/src/test/scala/unit/kafka/utils/TestUtils.scala: ## @@ -1872,7 +1872,7 @@ object TestUtils extends Logging { AdminClientUnitTestEnv.kafkaAdminClientNetw

Re: [PR] KAFKA-7632: Support Compression Levels (KIP-390) [kafka]

2024-05-17 Thread via GitHub
mimaison commented on code in PR #15516: URL: https://github.com/apache/kafka/pull/15516#discussion_r1605420401 ## core/src/test/scala/unit/kafka/log/LogValidatorTest.scala: ## @@ -1538,7 +1563,80 @@ class LogValidatorTest { assertEquals(e.recordErrors.size, 3) } - pr

Re: [PR] KAFKA-16626: Lazily convert subscribed topic names to topic ids [kafka]

2024-05-17 Thread via GitHub
jeffkbkim commented on PR #15970: URL: https://github.com/apache/kafka/pull/15970#issuecomment-2118184548 RangeAssignor baseline ``` Benchmark (memberCount) (partitionsToMemberRatio) (topicCount) Mode CntScoreError Units TargetAssignm

Re: [PR] KAFKA-16625: Reverse lookup map from topic partitions to members [kafka]

2024-05-17 Thread via GitHub
dajac commented on code in PR #15974: URL: https://github.com/apache/kafka/pull/15974#discussion_r1605415364 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroup.java: ## @@ -147,6 +147,11 @@ public static class DeadlineAndEpoch { */

[jira] [Commented] (KAFKA-16790) Calls to RemoteLogManager are made before it is configured

2024-05-17 Thread Muralidhar Basani (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847392#comment-17847392 ] Muralidhar Basani commented on KAFKA-16790: --- Even though brokerMetadataPublish

[jira] [Commented] (KAFKA-16692) InvalidRequestException: ADD_PARTITIONS_TO_TXN with version 4 which is not enabled when upgrading from kafka 3.5 to 3.6

2024-05-17 Thread Johnson Okorie (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847391#comment-17847391 ] Johnson Okorie commented on KAFKA-16692: Hi [~jolshan], I wanted to express my s

Re: [PR] KAFKA-16625: Reverse lookup map from topic partitions to members [kafka]

2024-05-17 Thread via GitHub
rreddy-22 commented on code in PR #15974: URL: https://github.com/apache/kafka/pull/15974#discussion_r1605382569 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroup.java: ## @@ -147,6 +147,11 @@ public static class DeadlineAndEpoch {

Re: [PR] MINOR: Fix rate metric spikes [kafka]

2024-05-17 Thread via GitHub
junrao commented on code in PR #15889: URL: https://github.com/apache/kafka/pull/15889#discussion_r1605350575 ## clients/src/main/java/org/apache/kafka/common/metrics/stats/SampledStat.java: ## @@ -110,40 +111,43 @@ public String toString() { protected void purgeObsoleteSam

Re: [PR] KAFKA-16654:Refactor kafka.test.annotation.Type and ClusterTestExtensions [kafka]

2024-05-17 Thread via GitHub
chia7712 commented on code in PR #15916: URL: https://github.com/apache/kafka/pull/15916#discussion_r1605371498 ## core/src/test/java/kafka/test/ClusterTestExtensionsTest.java: ## @@ -62,10 +63,10 @@ public class ClusterTestExtensionsTest { } // Static methods can ge

[jira] [Commented] (KAFKA-14517) Implement regex subscriptions

2024-05-17 Thread Phuc Hong Tran (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847386#comment-17847386 ] Phuc Hong Tran commented on KAFKA-14517: [~lianetm] I’m in the process of implem

[jira] [Commented] (KAFKA-16788) Resource leakage due to absence of close() call on connector start failure

2024-05-17 Thread Ashok (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847385#comment-17847385 ] Ashok commented on KAFKA-16788: --- [~gharris1727] Thanks. yes, I am working on the fix. I wi

[jira] [Commented] (KAFKA-16790) Calls to RemoteLogManager are made before it is configured

2024-05-17 Thread Muralidhar Basani (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847384#comment-17847384 ] Muralidhar Basani commented on KAFKA-16790: --- [~christo_lolov] can I look into

Re: [PR] MINOR: Refactor write timeout in CoordinatorRuntime [kafka]

2024-05-17 Thread via GitHub
chia7712 merged PR #15976: URL: https://github.com/apache/kafka/pull/15976 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[jira] [Commented] (KAFKA-16788) Resource leakage due to absence of close() call on connector start failure

2024-05-17 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847383#comment-17847383 ] Greg Harris commented on KAFKA-16788: - Nice find [~ashok89]! Are you interested in w

Re: [PR] KAFKA-16574: The metrics of LogCleaner disappear after reconfiguration [kafka]

2024-05-17 Thread via GitHub
chia7712 commented on code in PR #15863: URL: https://github.com/apache/kafka/pull/15863#discussion_r1605365153 ## core/src/main/scala/kafka/log/LogCleaner.scala: ## @@ -182,6 +183,27 @@ class LogCleaner(initialConfig: CleanerConfig, cleanerManager.removeMetrics() Review C

Re: [PR] KAFKA-16705 the flag "started" of RaftClusterInstance is false even though the cluster is started [kafka]

2024-05-17 Thread via GitHub
chia7712 commented on code in PR #15946: URL: https://github.com/apache/kafka/pull/15946#discussion_r1605361253 ## core/src/test/java/kafka/test/junit/RaftClusterInvocationContext.java: ## @@ -240,28 +215,61 @@ public void startBroker(int brokerId) { @Override

[jira] [Updated] (KAFKA-16790) Calls to RemoteLogManager are made before it is configured

2024-05-17 Thread Christo Lolov (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christo Lolov updated KAFKA-16790: -- Description: BrokerMetadataPublisher#onMetadataUpdate calls ReplicaManager#applyDelta (1) whi

[jira] [Updated] (KAFKA-16790) Calls to RemoteLogManager are made before it is configured

2024-05-17 Thread Christo Lolov (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christo Lolov updated KAFKA-16790: -- Description: BrokerMetadataPublisher#onMetadataUpdate calls ReplicaManager#applyDelta (1) whi

Re: [PR] KAFKA-15905 Restarts of MirrorCheckpointTask should not permanently i… [kafka]

2024-05-17 Thread via GitHub
prestona commented on PR #15910: URL: https://github.com/apache/kafka/pull/15910#issuecomment-2118027385 Hi @gharris1727, hopefully the latest commits address your review comments. Once again, really appreciate all your feedback and suggestions. -- This is an automated message from the Ap

[jira] [Created] (KAFKA-16790) Calls to RemoteLogManager are made before it is configured

2024-05-17 Thread Christo Lolov (Jira)
Christo Lolov created KAFKA-16790: - Summary: Calls to RemoteLogManager are made before it is configured Key: KAFKA-16790 URL: https://issues.apache.org/jira/browse/KAFKA-16790 Project: Kafka

Re: [PR] KAFKA-7632: Support Compression Levels (KIP-390) [kafka]

2024-05-17 Thread via GitHub
junrao commented on code in PR #15516: URL: https://github.com/apache/kafka/pull/15516#discussion_r1605313096 ## core/src/test/scala/unit/kafka/log/LogValidatorTest.scala: ## @@ -1538,7 +1563,80 @@ class LogValidatorTest { assertEquals(e.recordErrors.size, 3) } - priv

[PR] KAFKA-16252; Dynamic KRaft network manager and channel [kafka]

2024-05-17 Thread via GitHub
jsancio opened a new pull request, #15986: URL: https://github.com/apache/kafka/pull/15986 DRAFT ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including

[jira] [Resolved] (KAFKA-16763) Upgrade to scala 2.12.19 and scala 2.13.14

2024-05-17 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-16763. Fix Version/s: 3.8.0 Resolution: Fixed > Upgrade to scala 2.12.19 and scala 2.13.14

Re: [PR] KAFKA-16763: Upgrade to scala 2.12.19 and scala 2.13.14 [kafka]

2024-05-17 Thread via GitHub
chia7712 merged PR #15958: URL: https://github.com/apache/kafka/pull/15958 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[jira] [Commented] (KAFKA-14517) Implement regex subscriptions

2024-05-17 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847369#comment-17847369 ] Lianet Magrans commented on KAFKA-14517: hey [~phuctran], [~dajac] , I would lik

[jira] [Updated] (KAFKA-16774) fix flaky StreamThreadTest#shouldCloseAllTaskProducersOnCloseIfEosEnabled

2024-05-17 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai updated KAFKA-16774: --- Fix Version/s: 3.8.0 > fix flaky StreamThreadTest#shouldCloseAllTaskProducersOnCloseIfEosEna

[jira] [Assigned] (KAFKA-16758) Extend Consumer#close with option to leave the group or not

2024-05-17 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans reassigned KAFKA-16758: -- Assignee: Lianet Magrans > Extend Consumer#close with option to leave the group or no

[jira] [Commented] (KAFKA-15197) Flaky test MirrorConnectorsIntegrationExactlyOnceTest.testOffsetTranslationBehindReplicationFlow()

2024-05-17 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847366#comment-17847366 ] Chia-Ping Tsai commented on KAFKA-15197: It seems that behavior is expected. T

Re: [PR] KAFKA-16625: Reverse lookup map from topic partitions to members [kafka]

2024-05-17 Thread via GitHub
dajac commented on code in PR #15974: URL: https://github.com/apache/kafka/pull/15974#discussion_r1605223809 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/AbstractUniformAssignmentBuilder.java: ## @@ -65,4 +65,33 @@ protected static Set topicIdPa

[jira] [Commented] (KAFKA-16692) InvalidRequestException: ADD_PARTITIONS_TO_TXN with version 4 which is not enabled when upgrading from kafka 3.5 to 3.6

2024-05-17 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847352#comment-17847352 ] Justine Olshan commented on KAFKA-16692: [~soarez] I'd like to try to get this f

[jira] [Commented] (KAFKA-16333) Removed Deprecated methods KTable#join

2024-05-17 Thread Muralidhar Basani (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847348#comment-17847348 ] Muralidhar Basani commented on KAFKA-16333: --- Agree. thanks. > Removed Depreca

Re: [PR] KAFKA-15974: Enforce that event processing respects user-provided timeout [kafka]

2024-05-17 Thread via GitHub
lianetm commented on PR #15640: URL: https://github.com/apache/kafka/pull/15640#issuecomment-2117763853 Hey @cadonna, the tricky bit is that, for some events, the request managers do expire requests too, so in this flow you described: > The event is processed in the ApplicationEventHa

Re: [PR] [Draft] KAFKA-16764: New consumer should throw InvalidTopicException on poll when invalid topic in metadata [kafka]

2024-05-17 Thread via GitHub
appchemist commented on PR #15961: URL: https://github.com/apache/kafka/pull/15961#issuecomment-2117739675 @lianetm & @philipnee PTAL `testSubscriptionOnInvalidTopic` is still disabled for new consumer. I couldn't make the changes because `MockMetadataUpdater` is being used interna

[jira] [Comment Edited] (KAFKA-12679) Rebalancing a restoring or running task may cause directory livelocking with newly created task

2024-05-17 Thread Lucas Brutschy (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847334#comment-17847334 ] Lucas Brutschy edited comment on KAFKA-12679 at 5/17/24 2:26 PM: -

[jira] [Comment Edited] (KAFKA-12679) Rebalancing a restoring or running task may cause directory livelocking with newly created task

2024-05-17 Thread Lucas Brutschy (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847334#comment-17847334 ] Lucas Brutschy edited comment on KAFKA-12679 at 5/17/24 2:20 PM: -

Re: [PR] KAFKA-12399: Deprecate KafkaLog4jAppender [kafka]

2024-05-17 Thread via GitHub
mimaison commented on PR #15985: URL: https://github.com/apache/kafka/pull/15985#issuecomment-2117717230 @viktorsomogyi IIRC you volunteered to do the migration work to log4j2 a while back. Are you still able/willing to do so for Kafka 4.0? @jlprat As there's a chance 3.8 will be the

[jira] [Updated] (KAFKA-12679) Rebalancing a restoring or running task may cause directory livelocking with newly created task

2024-05-17 Thread Lucas Brutschy (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lucas Brutschy updated KAFKA-12679: --- Fix Version/s: 3.8.0 (was: 3.7.0) > Rebalancing a restoring or runnin

[jira] [Commented] (KAFKA-12679) Rebalancing a restoring or running task may cause directory livelocking with newly created task

2024-05-17 Thread Lucas Brutschy (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847334#comment-17847334 ] Lucas Brutschy commented on KAFKA-12679: Hey [~coltmcnealy-lh]. It is true that

[jira] [Resolved] (KAFKA-16762) SyncGroup API for upgrading ConsumerGroup

2024-05-17 Thread David Jacot (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-16762. - Fix Version/s: 3.8.0 Reviewer: David Jacot Resolution: Fixed > SyncGroup API for

[PR] KAFKA-12399: Deprecate KafkaLog4jAppender [kafka]

2024-05-17 Thread via GitHub
mimaison opened a new pull request, #15985: URL: https://github.com/apache/kafka/pull/15985 As per KIP-719, KafkaLog4jAppender is now deprecated and will be removed in Kafka 4.0. Users should migrate to the log4j2 Kafka Appender ### Committer Checklist (excluded from commit message)

Re: [PR] KAFKA-16762: SyncGroup API for upgrading ConsumerGroup [kafka]

2024-05-17 Thread via GitHub
dajac merged PR #15954: URL: https://github.com/apache/kafka/pull/15954 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

[jira] [Assigned] (KAFKA-16784) Migrate TopicBasedRemoteLogMetadataManagerMultipleSubscriptionsTest to new test infra

2024-05-17 Thread Chia-Ping Tsai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai reassigned KAFKA-16784: -- Assignee: PoAn Yang (was: Chia-Ping Tsai) > Migrate TopicBasedRemoteLogMetadataManag

[PR] KAFKA-16789: Fix thread leak detection for event handler threads [kafka]

2024-05-17 Thread via GitHub
gaurav-narula opened a new pull request, #15984: URL: https://github.com/apache/kafka/pull/15984 Updates `TestUtils::verifyNoUnexpectedThreads` to check for all event queue threads instead of the incorrect QuorumController thread. Also updated `KafkaEventQueueTest` to use try-with-res

Re: [PR] KAFKA-15746: KRaft support in ControllerMutationQuotaTest [kafka]

2024-05-17 Thread via GitHub
mimaison commented on PR #15038: URL: https://github.com/apache/kafka/pull/15038#issuecomment-2117661783 @linzihao1999 Can you follow up this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] KAFKA-16675: Refactored and new rebalance callbacks integration tests [kafka]

2024-05-17 Thread via GitHub
lianetm commented on PR #15965: URL: https://github.com/apache/kafka/pull/15965#issuecomment-2117652874 Thanks for the helpful comment @lucasbru , addressed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

Re: [PR] KAFKA-16675: Refactored and new rebalance callbacks integration tests [kafka]

2024-05-17 Thread via GitHub
lianetm commented on code in PR #15965: URL: https://github.com/apache/kafka/pull/15965#discussion_r1605041571 ## core/src/test/scala/integration/kafka/api/PlaintextConsumerCallbackTest.scala: ## @@ -84,29 +84,87 @@ class PlaintextConsumerCallbackTest extends AbstractConsumerTe

Re: [PR] KAFKA-16066: Upgrade apacheds to 2.0.0.AM27 With apache kerby [kafka]

2024-05-17 Thread via GitHub
mimaison commented on PR #15277: URL: https://github.com/apache/kafka/pull/15277#issuecomment-2117650240 @highluck Can you address the pending comments and rebase on trunk to resolve the conflicts? Thanks -- This is an automated message from the Apache Git Service. To respond to the messa

Re: [PR] KAFKA-16362: Fix type-unsafety in KStreamKStreamJoin caused by isLeftSide [kafka]

2024-05-17 Thread via GitHub
cadonna commented on code in PR #15601: URL: https://github.com/apache/kafka/pull/15601#discussion_r1605020116 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamKStreamLeftJoin.java: ## @@ -0,0 +1,72 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] KAFKA-16362: Fix type-unsafety in KStreamKStreamJoin caused by isLeftSide [kafka]

2024-05-17 Thread via GitHub
cadonna commented on code in PR #15601: URL: https://github.com/apache/kafka/pull/15601#discussion_r1604937635 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamKStreamJoin.java: ## @@ -187,17 +157,25 @@ public void process(final Record record) {

[jira] [Created] (KAFKA-16789) Thread leak detection checks for incorrect QuorumController thread name

2024-05-17 Thread Gaurav Narula (Jira)
Gaurav Narula created KAFKA-16789: - Summary: Thread leak detection checks for incorrect QuorumController thread name Key: KAFKA-16789 URL: https://issues.apache.org/jira/browse/KAFKA-16789 Project: Ka

[jira] [Assigned] (KAFKA-16789) Thread leak detection checks for incorrect QuorumController thread name

2024-05-17 Thread Gaurav Narula (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gaurav Narula reassigned KAFKA-16789: - Assignee: Gaurav Narula > Thread leak detection checks for incorrect QuorumController t

[jira] [Commented] (KAFKA-16448) Add Kafka Streams exception handler for exceptions occuring during processing (KIP-1033)

2024-05-17 Thread Muralidhar Basani (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847294#comment-17847294 ] Muralidhar Basani commented on KAFKA-16448: --- Thanks [~mjsax]  [~lmunoz] if th

[jira] [Commented] (KAFKA-16784) Migrate TopicBasedRemoteLogMetadataManagerMultipleSubscriptionsTest to new test infra

2024-05-17 Thread PoAn Yang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17847272#comment-17847272 ] PoAn Yang commented on KAFKA-16784: --- Hi [~chia7712], I'm interested in this issue. May

Re: [PR] KAFKA-15319: Upgrade rocksdb to fix CVE-2022-37434 [kafka]

2024-05-17 Thread via GitHub
lucasbru commented on PR #14216: URL: https://github.com/apache/kafka/pull/14216#issuecomment-2117519600 @mimaison -- Let me test this. Yes, I see that also the version of requests inside ducktape was reverted: https://github.com/confluentinc/ducktape/commit/29dcdbeb1a8048505ce1613eb9aad05b

[PR] KAFKA-16783: Migrate RemoteLogMetadataManagerTest to new test infra [kafka]

2024-05-17 Thread via GitHub
FrankYang0529 opened a new pull request, #15983: URL: https://github.com/apache/kafka/pull/15983 * Replace `TopicBasedRemoteLogMetadataManagerWrapperWithHarness` with `RemoteLogMetadataManagerTestUtils#builder` in `RemoteLogMetadataManagerTest`. * Use `ClusterTestExtention` for `RemoteLog

Re: [PR] KAFKA-15319: Upgrade rocksdb to fix CVE-2022-37434 [kafka]

2024-05-17 Thread via GitHub
mimaison commented on PR #14216: URL: https://github.com/apache/kafka/pull/14216#issuecomment-2117500715 Thanks @lucasbru ! I wonder if we also need to bump ducktape to 0.11.4 as it's seems previous versions depend on requests<2.31.0: https://ducktape.readthedocs.io/en/latest/changelog.h

Re: [PR] KAFKA-15319: Upgrade rocksdb to fix CVE-2022-37434 [kafka]

2024-05-17 Thread via GitHub
lucasbru commented on PR #14216: URL: https://github.com/apache/kafka/pull/14216#issuecomment-2117489283 @mimaison I remember I had problems running the system tests with that version, but I don't think I intended merging the version change. Sorry for that, I'll test the newer version and b

[PR] MINOR: use try-with resources in ClientMetricsManagerTest [kafka]

2024-05-17 Thread via GitHub
gaurav-narula opened a new pull request, #15982: URL: https://github.com/apache/kafka/pull/15982 Ensures we close resources even if an exception is thrown within a test. Also added an `@AfterAll` to ensure threads aren't leaked from the test class. ### Committer Checklist (excluded fr

Re: [PR] KAFKA-16197: Print Connect worker specific logs on poll timeout expiry [kafka]

2024-05-17 Thread via GitHub
vamossagar12 commented on PR #15305: URL: https://github.com/apache/kafka/pull/15305#issuecomment-2117312634 Thanks @gharris1727 for another round of review. I have made the changes. Please review when you have some time. -- This is an automated message from the Apache Git Service. To re

Re: [PR] KAFKA-16197: Print Connect worker specific logs on poll timeout expiry [kafka]

2024-05-17 Thread via GitHub
vamossagar12 commented on code in PR #15305: URL: https://github.com/apache/kafka/pull/15305#discussion_r1604768491 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/WorkerCoordinator.java: ## @@ -267,6 +268,20 @@ public String memberId() { r

Re: [PR] KAFKA-16197: Print Connect worker specific logs on poll timeout expiry [kafka]

2024-05-17 Thread via GitHub
vamossagar12 commented on code in PR #15305: URL: https://github.com/apache/kafka/pull/15305#discussion_r1604767686 ## connect/runtime/src/test/java/org/apache/kafka/connect/integration/ConnectWorkerIntegrationTest.java: ## @@ -850,6 +855,46 @@ public void testRequestTimeouts()

Re: [PR] KAFKA-16197: Print Connect worker specific logs on poll timeout expiry [kafka]

2024-05-17 Thread via GitHub
vamossagar12 commented on code in PR #15305: URL: https://github.com/apache/kafka/pull/15305#discussion_r1604759527 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/WorkerRebalanceListener.java: ## @@ -36,4 +37,15 @@ public interface WorkerRebalanceL

Re: [PR] KAFKA-16197: Print Connect worker specific logs on poll timeout expiry [kafka]

2024-05-17 Thread via GitHub
vamossagar12 commented on code in PR #15305: URL: https://github.com/apache/kafka/pull/15305#discussion_r1604757990 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/DistributedHerder.java: ## @@ -2704,6 +2704,11 @@ public void onRevoked(String leader

  1   2   >