Re: [PR] KAFKA-15853: Move KafkaConfig properties definition out of core [kafka]

2024-05-22 Thread via GitHub
nizhikov commented on PR #15501: URL: https://github.com/apache/kafka/pull/15501#issuecomment-2126366162 @OmniaGM Are you still working on moving KafkaConfig out of core? Can you, please, show me PR's you are currently working on regarding KafkaConfig? -- This is an automated message

Re: [PR] KAFKA-16373: KIP-1028: Adding code to support Apache Kafka Docker Official Images [kafka]

2024-05-22 Thread via GitHub
KrishVora01 commented on code in PR #16027: URL: https://github.com/apache/kafka/pull/16027#discussion_r1611093413 ## docker/generate_kafka_pr_template.py: ## @@ -0,0 +1,100 @@ +#!/usr/bin/env python + +# Licensed to the Apache Software Foundation (ASF) under one or more +# cont

Re: [PR] KAFKA-16373: KIP-1028: Adding code to support Apache Kafka Docker Official Images [kafka]

2024-05-22 Thread via GitHub
KrishVora01 commented on code in PR #16027: URL: https://github.com/apache/kafka/pull/16027#discussion_r1611093207 ## docker/docker_official_image_build_test.py: ## @@ -0,0 +1,90 @@ +#!/usr/bin/env python + +# Licensed to the Apache Software Foundation (ASF) under one or more +#

Re: [PR] KAFKA-16373: KIP-1028: Adding code to support Apache Kafka Docker Official Images [kafka]

2024-05-22 Thread via GitHub
KrishVora01 commented on code in PR #16027: URL: https://github.com/apache/kafka/pull/16027#discussion_r1611092562 ## .github/workflows/prepare_docker_official_image_source.yml: ## @@ -0,0 +1,53 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contribu

Re: [PR] KAFKA-16373: KIP-1028: Adding code to support Apache Kafka Docker Official Images [kafka]

2024-05-22 Thread via GitHub
KrishVora01 commented on code in PR #16027: URL: https://github.com/apache/kafka/pull/16027#discussion_r1611091209 ## .github/workflows/docker_official_image_build_and_test.yml: ## @@ -0,0 +1,66 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contribu

Re: [PR] KAFKA-16652: add unit test for ClusterTemplate offering zero ClusterConfig [kafka]

2024-05-22 Thread via GitHub
TaiJuWu commented on code in PR #15862: URL: https://github.com/apache/kafka/pull/15862#discussion_r1611087227 ## core/src/test/java/kafka/test/junit/ClusterTestExtensionsUnitTest.java: ## @@ -33,16 +31,22 @@ public class ClusterTestExtensionsUnitTest { void testProcessClus

Re: [PR] KAFKA-16373: KIP-1028: Adding code to support Apache Kafka Docker Official Images [kafka]

2024-05-22 Thread via GitHub
VedarthConfluent commented on code in PR #16027: URL: https://github.com/apache/kafka/pull/16027#discussion_r1611077695 ## docker/generate_kafka_pr_template.py: ## @@ -0,0 +1,100 @@ +#!/usr/bin/env python + +# Licensed to the Apache Software Foundation (ASF) under one or more +#

Re: [PR] KAFKA-16452: Don't throw OOORE when converting the offset to metadata [kafka]

2024-05-22 Thread via GitHub
showuon commented on code in PR #15825: URL: https://github.com/apache/kafka/pull/15825#discussion_r1611046740 ## core/src/test/scala/unit/kafka/log/LogSegmentTest.scala: ## @@ -144,6 +144,35 @@ class LogSegmentTest { checkEquals(ms2.records.iterator, read.records.records.i

[jira] [Resolved] (KAFKA-16793) Heartbeat API for upgrading ConsumerGroup

2024-05-22 Thread David Jacot (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-16793. - Fix Version/s: 3.8.0 Reviewer: David Jacot Resolution: Fixed > Heartbeat API for

Re: [PR] KAFKA-16793: Heartbeat API for upgrading ConsumerGroup [kafka]

2024-05-22 Thread via GitHub
dajac merged PR #15988: URL: https://github.com/apache/kafka/pull/15988 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

Re: [PR] KAFKA-16793: Heartbeat API for upgrading ConsumerGroup [kafka]

2024-05-22 Thread via GitHub
dajac commented on PR #15988: URL: https://github.com/apache/kafka/pull/15988#issuecomment-2126326606 All the `org.apache.kafka.streams.processor.internals.assignment.RackAwareTaskAssignorTest` related failures are due to https://github.com/apache/kafka/pull/15972. I will merge it. -- T

Re: [PR] MINOR: fix flaky testRecordThreadIdleRatio [kafka]

2024-05-22 Thread via GitHub
dajac merged PR #15987: URL: https://github.com/apache/kafka/pull/15987 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

[jira] [Created] (KAFKA-16825) CVE vulnerabilities in Jetty and netty

2024-05-22 Thread mooner (Jira)
mooner created KAFKA-16825: -- Summary: CVE vulnerabilities in Jetty and netty Key: KAFKA-16825 URL: https://issues.apache.org/jira/browse/KAFKA-16825 Project: Kafka Issue Type: Task Affects Versi

Re: [PR] KAFKA-15045: (KIP-924 pt. 5) Add rack information to ApplicationState [kafka]

2024-05-22 Thread via GitHub
dajac commented on PR #15972: URL: https://github.com/apache/kafka/pull/15972#issuecomment-2126312436 Hey @ableegoldman @apourchet, I see new failures in trunk that seems to be related to this PR. The last build of this PR had 100+ failures: https://ci-builds.apache.org/blue/organizations/j

Re: [PR] KAFKA-16373: KIP-1028: Adding code to support Apache Kafka Docker Official Images [kafka]

2024-05-22 Thread via GitHub
VedarthConfluent commented on code in PR #16027: URL: https://github.com/apache/kafka/pull/16027#discussion_r1611056103 ## docker/docker_official_images/.gitignore: ## Review Comment: Can consider using .gitkeep if empty folder is needed -- This is an automated message

Re: [PR] KAFKA-16373: KIP-1028: Adding code to support Apache Kafka Docker Official Images [kafka]

2024-05-22 Thread via GitHub
VedarthConfluent commented on code in PR #16027: URL: https://github.com/apache/kafka/pull/16027#discussion_r1611048213 ## docker/docker_official_images/.gitignore: ## Review Comment: Not needed, we can remove this -- This is an automated message from the Apache Git Ser

Re: [PR] KAFKA-16373: KIP-1028: Adding code to support Apache Kafka Docker Official Images [kafka]

2024-05-22 Thread via GitHub
VedarthConfluent commented on code in PR #16027: URL: https://github.com/apache/kafka/pull/16027#discussion_r1611047289 ## docker/docker_official_image_build_test.py: ## @@ -0,0 +1,90 @@ +#!/usr/bin/env python + +# Licensed to the Apache Software Foundation (ASF) under one or mo

Re: [PR] KAFKA-16373: KIP-1028: Adding code to support Apache Kafka Docker Official Images [kafka]

2024-05-22 Thread via GitHub
VedarthConfluent commented on code in PR #16027: URL: https://github.com/apache/kafka/pull/16027#discussion_r1611042138 ## .github/workflows/prepare_docker_official_image_source.yml: ## @@ -0,0 +1,53 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# con

Re: [PR] KAFKA-16373: KIP-1028: Adding code to support Apache Kafka Docker Official Images [kafka]

2024-05-22 Thread via GitHub
VedarthConfluent commented on code in PR #16027: URL: https://github.com/apache/kafka/pull/16027#discussion_r1611041547 ## .github/workflows/prepare_docker_official_image_source.yml: ## @@ -0,0 +1,53 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# con

Re: [PR] KAFKA-16373: KIP-1028: Adding code to support Apache Kafka Docker Official Images [kafka]

2024-05-22 Thread via GitHub
VedarthConfluent commented on code in PR #16027: URL: https://github.com/apache/kafka/pull/16027#discussion_r1611040507 ## .github/workflows/docker_official_image_build_and_test.yml: ## @@ -0,0 +1,66 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# con

[jira] [Assigned] (KAFKA-16257) SchemaProjector should be extensible to logical types

2024-05-22 Thread Fan Yang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fan Yang reassigned KAFKA-16257: Assignee: Fan Yang > SchemaProjector should be extensible to logical types >

[jira] [Commented] (KAFKA-16820) Kafka Broker fails to connect to Kraft Controller with no DNS matching

2024-05-22 Thread Vikash Mishra (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17848821#comment-17848821 ] Vikash Mishra commented on KAFKA-16820: --- Despite the fact that brokers are provide

[PR] Improve logical type compatibility in SchemaProjector [kafka]

2024-05-22 Thread via GitHub
fanyang opened a new pull request, #16035: URL: https://github.com/apache/kafka/pull/16035 This PR improved logical type compatibility in SchemaProjector. Decimal type with higher scale is compatible with one with same or lower scale. Timestamp type is compatible with Date and Time typ

Re: [PR] KAFKA-15541: Add iterator-duration metrics [kafka]

2024-05-22 Thread via GitHub
mjsax merged PR #16028: URL: https://github.com/apache/kafka/pull/16028 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

Re: [PR] KAFKA-15541: Add iterator-duration metrics [kafka]

2024-05-22 Thread via GitHub
mjsax commented on code in PR #16028: URL: https://github.com/apache/kafka/pull/16028#discussion_r1610944460 ## streams/src/main/java/org/apache/kafka/streams/state/internals/metrics/StateStoreMetrics.java: ## @@ -149,6 +149,14 @@ private StateStoreMetrics() {} private stat

Re: [PR] KAFKA-14412: Add statestore.uncommitted.max.bytes [kafka]

2024-05-22 Thread via GitHub
github-actions[bot] commented on PR #15264: URL: https://github.com/apache/kafka/pull/15264#issuecomment-2126168320 This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please ask a committer for review. If the PR has merge

Re: [PR] KAFKA-16452: Don't throw OOORE when converting the offset to metadata [kafka]

2024-05-22 Thread via GitHub
kamalcph commented on code in PR #15825: URL: https://github.com/apache/kafka/pull/15825#discussion_r1610893769 ## core/src/test/scala/integration/kafka/server/DelayedFetchTest.scala: ## @@ -164,18 +169,71 @@ class DelayedFetchTest { assertTrue(delayedFetch.tryComplete())

Re: [PR] KAFKA-16452: Don't throw OOORE when converting the offset to metadata [kafka]

2024-05-22 Thread via GitHub
kamalcph commented on code in PR #15825: URL: https://github.com/apache/kafka/pull/15825#discussion_r1610893769 ## core/src/test/scala/integration/kafka/server/DelayedFetchTest.scala: ## @@ -164,18 +169,71 @@ class DelayedFetchTest { assertTrue(delayedFetch.tryComplete())

[jira] [Commented] (KAFKA-16811) Punctuate Ratio metric almost impossible to track

2024-05-22 Thread Rohan Desai (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17848797#comment-17848797 ] Rohan Desai commented on KAFKA-16811: - Alternatively, or perhaps in addition to a pr

Re: [PR] KAFKA-16452: Don't throw OOORE when converting the offset to metadata [kafka]

2024-05-22 Thread via GitHub
kamalcph commented on code in PR #15825: URL: https://github.com/apache/kafka/pull/15825#discussion_r1610895466 ## core/src/main/scala/kafka/server/DelayedFetch.scala: ## @@ -91,19 +91,23 @@ class DelayedFetch( // Go directly to the check for Case G if the message o

Re: [PR] KAFKA-16452: Don't throw OOORE when converting the offset to metadata [kafka]

2024-05-22 Thread via GitHub
kamalcph commented on code in PR #15825: URL: https://github.com/apache/kafka/pull/15825#discussion_r1610893769 ## core/src/test/scala/integration/kafka/server/DelayedFetchTest.scala: ## @@ -164,18 +169,71 @@ class DelayedFetchTest { assertTrue(delayedFetch.tryComplete())

[jira] [Commented] (KAFKA-16160) AsyncKafkaConsumer is trying to connect to a disconnected node in a tight loop

2024-05-22 Thread Phuc Hong Tran (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17848791#comment-17848791 ] Phuc Hong Tran commented on KAFKA-16160: [~pnee] I understand > AsyncKafkaCons

[jira] [Commented] (KAFKA-14517) Implement regex subscriptions

2024-05-22 Thread Phuc Hong Tran (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17848792#comment-17848792 ] Phuc Hong Tran commented on KAFKA-14517: Thanks again [~lianetm] > Implement r

Re: [PR] KAFKA-16771 First log directory printed twice when formatting storage [kafka]

2024-05-22 Thread via GitHub
gongxuanzhang commented on code in PR #16010: URL: https://github.com/apache/kafka/pull/16010#discussion_r1610875448 ## core/src/test/scala/unit/kafka/tools/StorageToolTest.scala: ## @@ -488,4 +488,26 @@ Found problem: assertEquals(1, exitStatus) } } + + @Test +

Re: [PR] KAFKA-15541: Add num-open-iterators metric [kafka]

2024-05-22 Thread via GitHub
mjsax commented on PR #15975: URL: https://github.com/apache/kafka/pull/15975#issuecomment-2126049466 Makes sense. Could you file a Jira for the KIP and the follow up cleanup for KS code to use it :) -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] KAFKA-16709: abortAndPauseCleaning only when future log is not existed [kafka]

2024-05-22 Thread via GitHub
showuon commented on code in PR #15951: URL: https://github.com/apache/kafka/pull/15951#discussion_r1610854305 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -2114,16 +2114,12 @@ class ReplicaManager(val config: KafkaConfig, partition.log.foreach { _ =>

Re: [PR] KAFKA-5072[WIP]: Kafka topics should allow custom metadata configs within some config namespace [kafka]

2024-05-22 Thread via GitHub
andreyolv commented on PR #2873: URL: https://github.com/apache/kafka/pull/2873#issuecomment-2125996595 Extremely important feature for governance. Any release forecast? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] KAFKA-15045: (KIP-924 pt. 10) Topic partition rack annotation simplified [kafka]

2024-05-22 Thread via GitHub
apourchet commented on code in PR #16034: URL: https://github.com/apache/kafka/pull/16034#discussion_r1610816330 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamsPartitionAssignor.java: ## @@ -513,50 +515,45 @@ private ApplicationState buildApplicatio

Re: [PR] KAFKA-16598 Mirgrate `ResetConsumerGroupOffsetTest` to new test infra [kafka]

2024-05-22 Thread via GitHub
m1a2st commented on code in PR #15779: URL: https://github.com/apache/kafka/pull/15779#discussion_r1610809626 ## tools/src/test/java/org/apache/kafka/tools/consumer/group/ResetConsumerGroupOffsetTest.java: ## @@ -62,506 +86,756 @@ * - scope=topics+partitions, scenario=to-earli

[jira] [Commented] (KAFKA-16814) KRaft broker cannot startup when `partition.metadata` is missing

2024-05-22 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17848781#comment-17848781 ] Luke Chen commented on KAFKA-16814: --- [~muralibasani] , no, it's not related to remote

Re: [PR] KAFKA-9228: Restart tasks on runtime-only connector config changes [kafka]

2024-05-22 Thread via GitHub
gharris1727 commented on code in PR #16001: URL: https://github.com/apache/kafka/pull/16001#discussion_r1610727814 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/AbstractHerder.java: ## @@ -1039,7 +1039,12 @@ public static List> reverseTransform(String connNa

Re: [PR] KAFKA-16764: New consumer should throw InvalidTopicException on poll when invalid topic in metadata [kafka]

2024-05-22 Thread via GitHub
appchemist commented on PR #15961: URL: https://github.com/apache/kafka/pull/15961#issuecomment-2125950049 @lianetm I got it, Thank you! @kirktrue Right! I think what you said is simple and more intuitive. -- This is an automated message from the Apache Git Service. To respond to the me

[jira] [Updated] (KAFKA-15630) Improve documentation of offset.lag.max

2024-05-22 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Harris updated KAFKA-15630: Labels: newbie (was: ) > Improve documentation of offset.lag.max > --

[jira] [Updated] (KAFKA-15630) Improve documentation of offset.lag.max

2024-05-22 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Harris updated KAFKA-15630: Description: It would be good to expand on the role of this configuration on offset translation a

[jira] [Commented] (KAFKA-16798) Mirrormaker2 dedicated mode - sync.group.offsets.interval not working

2024-05-22 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17848770#comment-17848770 ] Greg Harris commented on KAFKA-16798: - Hi [~sektor.coder] There's already a ticket t

Re: [PR] KAFKA-15045: (KIP-924 pt. 10) Topic partition rack annotation simplified [kafka]

2024-05-22 Thread via GitHub
ableegoldman commented on code in PR #16034: URL: https://github.com/apache/kafka/pull/16034#discussion_r1610769106 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/assignment/RackUtils.java: ## @@ -38,26 +40,37 @@ public final class RackUtils { priva

Re: [PR] KAFKA-15045: (KIP-924 pt. 9) TaskAssignmentUtils implementation of optimizeRackAwareActiveTasks [kafka]

2024-05-22 Thread via GitHub
apourchet commented on code in PR #16033: URL: https://github.com/apache/kafka/pull/16033#discussion_r1610768911 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/assignment/RackAwareTaskAssignor.java: ## @@ -486,12 +486,12 @@ public long optimizeStandbyTasks

Re: [PR] KAFKA-15045: (KIP-924 pt. 9) TaskAssignmentUtils implementation of optimizeRackAwareActiveTasks [kafka]

2024-05-22 Thread via GitHub
apourchet commented on code in PR #16033: URL: https://github.com/apache/kafka/pull/16033#discussion_r1610767256 ## streams/src/main/java/org/apache/kafka/streams/processor/assignment/TaskInfo.java: ## @@ -50,6 +50,12 @@ public interface TaskInfo { */ Set stateStoreNa

Re: [PR] KAFKA-15045: (KIP-924 pt. 8) Added TopicPartitionAssignmentInfo [kafka]

2024-05-22 Thread via GitHub
ableegoldman merged PR #16024: URL: https://github.com/apache/kafka/pull/16024 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.ap

[PR] KAFKA-15045: (KIP-924 pt. 10) Topic partition rack annotation simplified [kafka]

2024-05-22 Thread via GitHub
apourchet opened a new pull request, #16034: URL: https://github.com/apache/kafka/pull/16034 This PR uses the new TaskTopicPartition structure to simplify the build process for the ApplicationState, which is the input to the new TaskAssignor#assign call. -- This is an automated messa

[jira] [Comment Edited] (KAFKA-16798) Mirrormaker2 dedicated mode - sync.group.offsets.interval not working

2024-05-22 Thread Thanos Athanasopoulos (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17848767#comment-17848767 ] Thanos Athanasopoulos edited comment on KAFKA-16798 at 5/22/24 10:44 PM: -

[jira] [Comment Edited] (KAFKA-16798) Mirrormaker2 dedicated mode - sync.group.offsets.interval not working

2024-05-22 Thread Thanos Athanasopoulos (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17848767#comment-17848767 ] Thanos Athanasopoulos edited comment on KAFKA-16798 at 5/22/24 10:39 PM: -

[jira] [Comment Edited] (KAFKA-16798) Mirrormaker2 dedicated mode - sync.group.offsets.interval not working

2024-05-22 Thread Thanos Athanasopoulos (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17848767#comment-17848767 ] Thanos Athanasopoulos edited comment on KAFKA-16798 at 5/22/24 10:38 PM: -

[jira] [Commented] (KAFKA-16798) Mirrormaker2 dedicated mode - sync.group.offsets.interval not working

2024-05-22 Thread Thanos Athanasopoulos (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17848767#comment-17848767 ] Thanos Athanasopoulos commented on KAFKA-16798: --- [~gharris1727] that was t

Re: [PR] KAFKA-16793: Heartbeat API for upgrading ConsumerGroup [kafka]

2024-05-22 Thread via GitHub
dongnuo123 commented on code in PR #15988: URL: https://github.com/apache/kafka/pull/15988#discussion_r1610735381 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -12401,6 +12436,363 @@ public void testClassicGroupSyncTo

[jira] [Assigned] (KAFKA-15284) Implement ConsumerGroupProtocolVersionResolver to determine consumer group protocol

2024-05-22 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True reassigned KAFKA-15284: - Assignee: (was: Kirk True) > Implement ConsumerGroupProtocolVersionResolver to determine co

Re: [PR] KAFKA-16793: Heartbeat API for upgrading ConsumerGroup [kafka]

2024-05-22 Thread via GitHub
dongnuo123 commented on code in PR #15988: URL: https://github.com/apache/kafka/pull/15988#discussion_r1610729396 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -12143,7 +12183,6 @@ public void testClassicGroupSyncToCo

Re: [PR] KAFKA-16793: Heartbeat API for upgrading ConsumerGroup [kafka]

2024-05-22 Thread via GitHub
dongnuo123 commented on code in PR #15988: URL: https://github.com/apache/kafka/pull/15988#discussion_r1610728790 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -4274,6 +4343,81 @@ private void validateClassicGroupHeartbeat

Re: [PR] KAFKA-9228: Restart tasks on runtime-only connector config changes [kafka]

2024-05-22 Thread via GitHub
C0urante commented on code in PR #16001: URL: https://github.com/apache/kafka/pull/16001#discussion_r1610679966 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/ConfigHash.java: ## @@ -0,0 +1,245 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] KAFKA-9228: Restart tasks on runtime-only connector config changes [kafka]

2024-05-22 Thread via GitHub
C0urante commented on PR #16001: URL: https://github.com/apache/kafka/pull/16001#issuecomment-2125789945 After some offline discussion with @gharris1727 we realized there was a potential for infinite rebalance loops with the previous implementation when config providers resolved values diff

[jira] [Assigned] (KAFKA-16815) Handle FencedInstanceId on heartbeat for new consumer

2024-05-22 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans reassigned KAFKA-16815: -- Assignee: Lianet Magrans > Handle FencedInstanceId on heartbeat for new consumer > --

[jira] [Resolved] (KAFKA-16160) AsyncKafkaConsumer is trying to connect to a disconnected node in a tight loop

2024-05-22 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee resolved KAFKA-16160. Resolution: Cannot Reproduce > AsyncKafkaConsumer is trying to connect to a disconnected node in a

[jira] [Commented] (KAFKA-16160) AsyncKafkaConsumer is trying to connect to a disconnected node in a tight loop

2024-05-22 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17848748#comment-17848748 ] Philip Nee commented on KAFKA-16160: Hey [~phuctran] - I'm going to close this issue

[jira] [Updated] (KAFKA-16160) AsyncKafkaConsumer is trying to connect to a disconnected node in a tight loop

2024-05-22 Thread Jon Chiu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Chiu updated KAFKA-16160: - Priority: Major (was: Blocker) > AsyncKafkaConsumer is trying to connect to a disconnected node in a ti

[jira] [Assigned] (KAFKA-16623) KafkaAsyncConsumer system tests warn about revoking partitions that weren't previously assigned

2024-05-22 Thread Jon Chiu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Chiu reassigned KAFKA-16623: Assignee: Lianet Magrans (was: Kirk True) > KafkaAsyncConsumer system tests warn about revoking

Re: [PR] MINOR: Fix rate metric spikes [kafka]

2024-05-22 Thread via GitHub
emitskevich-blp commented on code in PR #15889: URL: https://github.com/apache/kafka/pull/15889#discussion_r1610656953 ## clients/src/main/java/org/apache/kafka/common/metrics/stats/SampledStat.java: ## @@ -50,10 +50,13 @@ public void record(MetricConfig config, double value, lo

Re: [PR] MINOR: Fix rate metric spikes [kafka]

2024-05-22 Thread via GitHub
emitskevich-blp commented on code in PR #15889: URL: https://github.com/apache/kafka/pull/15889#discussion_r1610656953 ## clients/src/main/java/org/apache/kafka/common/metrics/stats/SampledStat.java: ## @@ -50,10 +50,13 @@ public void record(MetricConfig config, double value, lo

[jira] [Updated] (KAFKA-16639) AsyncKafkaConsumer#close does not send heartbeat to leave group

2024-05-22 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-16639: -- Priority: Critical (was: Major) > AsyncKafkaConsumer#close does not send heartbeat to leave group > -

Re: [PR] MINOR: Fix rate metric spikes [kafka]

2024-05-22 Thread via GitHub
emitskevich-blp commented on code in PR #15889: URL: https://github.com/apache/kafka/pull/15889#discussion_r1610653813 ## clients/src/test/java/org/apache/kafka/common/metrics/stats/SampledStatTest.java: ## @@ -0,0 +1,110 @@ +/* + * Licensed to the Apache Software Foundation (AS

[PR] KAFKA-15045: (KIP-924 pt. 9) TaskAssignmentUtils implementation of optimizeRackAwareActiveTasks [kafka]

2024-05-22 Thread via GitHub
apourchet opened a new pull request, #16033: URL: https://github.com/apache/kafka/pull/16033 This PR implements the rack aware optimization of active tasks that can be used by the assignors themselves. It takes in the full output of the assignment and tries to reorganize it so as to minimiz

Re: [PR] KAFKA-16252; Dynamic KRaft network manager and channel [kafka]

2024-05-22 Thread via GitHub
jsancio commented on code in PR #15986: URL: https://github.com/apache/kafka/pull/15986#discussion_r1610641974 ## core/src/test/scala/unit/kafka/server/KafkaConfigTest.scala: ## @@ -1366,6 +1368,26 @@ class KafkaConfigTest { assertEquals(expectedVoters, addresses) } +

[jira] [Created] (KAFKA-16824) Utils.getHost and Utils.getPort do not catch a lot of invalid host and ports

2024-05-22 Thread Jira
José Armando García Sancio created KAFKA-16824: -- Summary: Utils.getHost and Utils.getPort do not catch a lot of invalid host and ports Key: KAFKA-16824 URL: https://issues.apache.org/jira/browse/KAFKA

[PR] KAFKA-16737: Enable unit tests for new consumer & cleanup TODOs [kafka]

2024-05-22 Thread via GitHub
lianetm opened a new pull request, #16032: URL: https://github.com/apache/kafka/pull/16032 KafkaConsumerTest contained lots of unit tests that were not enabled for the new consumer, with TODOs for verifying if they cold be enabled. This PR enables all the unit tests that could be applied/pa

Re: [PR] KAFKA-8206: Allow client to rebootstrap [kafka]

2024-05-22 Thread via GitHub
gharris1727 commented on code in PR #13277: URL: https://github.com/apache/kafka/pull/13277#discussion_r1610617003 ## clients/src/main/java/org/apache/kafka/clients/CommonClientConfigs.java: ## @@ -219,6 +219,19 @@ public class CommonClientConfigs { public static final Stri

[jira] [Created] (KAFKA-16823) Extract LegacyConsumer-specific unit tests from generic KafkaConsumerTest

2024-05-22 Thread Lianet Magrans (Jira)
Lianet Magrans created KAFKA-16823: -- Summary: Extract LegacyConsumer-specific unit tests from generic KafkaConsumerTest Key: KAFKA-16823 URL: https://issues.apache.org/jira/browse/KAFKA-16823 Projec

Re: [PR] KAFKA-10551: Add topic id support to produce request and response [kafka]

2024-05-22 Thread via GitHub
jolshan commented on code in PR #15968: URL: https://github.com/apache/kafka/pull/15968#discussion_r1610605263 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -1361,10 +1366,10 @@ class ReplicaManager(val config: KafkaConfig, */ private def appendToLocalLo

Re: [PR] MINOR: Fix rate metric spikes [kafka]

2024-05-22 Thread via GitHub
junrao commented on code in PR #15889: URL: https://github.com/apache/kafka/pull/15889#discussion_r1610602287 ## clients/src/main/java/org/apache/kafka/common/metrics/stats/SampledStat.java: ## @@ -106,44 +109,51 @@ public String toString() { public abstract double combin

Re: [PR] KAFKA-15045: (KIP-924 pt. 7) Simplify requirements for rack aware graphs [kafka]

2024-05-22 Thread via GitHub
ableegoldman merged PR #16004: URL: https://github.com/apache/kafka/pull/16004 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.ap

Re: [PR] KAFKA-15045: (KIP-924 pt. 5) Add rack information to ApplicationState [kafka]

2024-05-22 Thread via GitHub
ableegoldman merged PR #15972: URL: https://github.com/apache/kafka/pull/15972 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.ap

[jira] [Updated] (KAFKA-16819) CoordinatorRequestManager seems to return 0ms during the coordinator discovery

2024-05-22 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-16819: -- Labels: consumer-threading-refactor (was: ) > CoordinatorRequestManager seems to return 0ms during th

[jira] [Updated] (KAFKA-16819) CoordinatorRequestManager seems to return 0ms during the coordinator discovery

2024-05-22 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-16819: -- Component/s: clients > CoordinatorRequestManager seems to return 0ms during the coordinator discovery

Re: [PR] KAFKA-16452: Don't throw OOORE when converting the offset to metadata [kafka]

2024-05-22 Thread via GitHub
junrao commented on code in PR #15825: URL: https://github.com/apache/kafka/pull/15825#discussion_r1610572095 ## core/src/main/scala/kafka/server/DelayedFetch.scala: ## @@ -91,19 +91,23 @@ class DelayedFetch( // Go directly to the check for Case G if the message off

Re: [PR] KAFKA-15713: KRaft support in AclCommandTest [kafka]

2024-05-22 Thread via GitHub
chia7712 commented on code in PR #15830: URL: https://github.com/apache/kafka/pull/15830#discussion_r1610529172 ## core/src/test/scala/unit/kafka/admin/AclCommandTest.scala: ## @@ -122,19 +128,27 @@ class AclCommandTest extends QuorumTestHarness with Logging { super.tearDo

[jira] [Created] (KAFKA-16822) Abstract consumer group in coordinator to share functionality with share group

2024-05-22 Thread Apoorv Mittal (Jira)
Apoorv Mittal created KAFKA-16822: - Summary: Abstract consumer group in coordinator to share functionality with share group Key: KAFKA-16822 URL: https://issues.apache.org/jira/browse/KAFKA-16822 Proj

[jira] [Updated] (KAFKA-16766) New consumer offsetsForTimes timeout exception does not have the proper message

2024-05-22 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans updated KAFKA-16766: --- Description: If a call to consumer.offsetsForTimes times out, the new AsyncKafkaConsumer wi

Re: [PR] MINOR: Fix rate metric spikes [kafka]

2024-05-22 Thread via GitHub
chia7712 commented on code in PR #15889: URL: https://github.com/apache/kafka/pull/15889#discussion_r1610336329 ## clients/src/main/java/org/apache/kafka/common/metrics/stats/SampledStat.java: ## @@ -40,7 +40,7 @@ public abstract class SampledStat implements MeasurableStat {

Re: [PR] KAFKA-16308 [1/N]: Create FeatureVersion interface and add `--feature` flag and handling to StorageTool [kafka]

2024-05-22 Thread via GitHub
jolshan commented on code in PR #15685: URL: https://github.com/apache/kafka/pull/15685#discussion_r1610507371 ## server-common/src/main/java/org/apache/kafka/server/common/TestFeatureVersion.java: ## @@ -0,0 +1,56 @@ +/* + * Licensed to the Apache Software Foundation (ASF) unde

[jira] [Created] (KAFKA-16821) Create a new interface to store member metadata

2024-05-22 Thread Ritika Reddy (Jira)
Ritika Reddy created KAFKA-16821: Summary: Create a new interface to store member metadata Key: KAFKA-16821 URL: https://issues.apache.org/jira/browse/KAFKA-16821 Project: Kafka Issue Type: S

Re: [PR] KAFKA-16793: Heartbeat API for upgrading ConsumerGroup [kafka]

2024-05-22 Thread via GitHub
jeffkbkim commented on code in PR #15988: URL: https://github.com/apache/kafka/pull/15988#discussion_r1610481770 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -12401,6 +12436,363 @@ public void testClassicGroupSyncToC

Re: [PR] KAFKA-16308 [1/N]: Create FeatureVersion interface and add `--feature` flag and handling to StorageTool [kafka]

2024-05-22 Thread via GitHub
jolshan commented on code in PR #15685: URL: https://github.com/apache/kafka/pull/15685#discussion_r1610472137 ## server-common/src/main/java/org/apache/kafka/server/common/Features.java: ## @@ -64,14 +63,16 @@ public enum Features { PRODUCTION_FEATURES = Arrays.strea

Re: [PR] KAFKA-16308 [1/N]: Create FeatureVersion interface and add `--feature` flag and handling to StorageTool [kafka]

2024-05-22 Thread via GitHub
jolshan commented on code in PR #15685: URL: https://github.com/apache/kafka/pull/15685#discussion_r1610471028 ## server-common/src/main/java/org/apache/kafka/server/common/FeatureVersion.java: ## @@ -0,0 +1,49 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under on

Re: [PR] KAFKA-16308 [1/N]: Create FeatureVersion interface and add `--feature` flag and handling to StorageTool [kafka]

2024-05-22 Thread via GitHub
jolshan commented on code in PR #15685: URL: https://github.com/apache/kafka/pull/15685#discussion_r1610471028 ## server-common/src/main/java/org/apache/kafka/server/common/FeatureVersion.java: ## @@ -0,0 +1,49 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under on

Re: [PR] KAFKA-16793: Heartbeat API for upgrading ConsumerGroup [kafka]

2024-05-22 Thread via GitHub
jeffkbkim commented on code in PR #15988: URL: https://github.com/apache/kafka/pull/15988#discussion_r1610471371 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -12401,6 +12436,363 @@ public void testClassicGroupSyncToC

[jira] [Updated] (KAFKA-16820) Kafka Broker fails to connect to Kraft Controller with no DNS matching

2024-05-22 Thread Arushi Helms (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arushi Helms updated KAFKA-16820: - Description:   We are migrating our Kafka cluster from zookeeper to Kraft mode. We are running

Re: [PR] Update RemoteLogManager configuration in broker server - KAFKA-16790 [kafka]

2024-05-22 Thread via GitHub
muralibasani commented on PR #16005: URL: https://github.com/apache/kafka/pull/16005#issuecomment-2125492819 @nikramakrishnan I think calling applyDelta on replicaManager wouldn't invoke configure on rlmm. Probably we should start mocking from BrokerServer.startup method, and create a t

Re: [PR] KAFKA-16516: Fix the controller node provider for broker to control channel [kafka]

2024-05-22 Thread via GitHub
jsancio commented on code in PR #16008: URL: https://github.com/apache/kafka/pull/16008#discussion_r1610457959 ## core/src/main/scala/kafka/server/NodeToControllerChannelManager.scala: ## @@ -120,15 +118,15 @@ object RaftControllerNodeProvider { */ class RaftControllerNodePro

[jira] [Updated] (KAFKA-16820) Kafka Broker fails to connect to Kraft Controller with no DNS matching

2024-05-22 Thread Arushi Helms (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arushi Helms updated KAFKA-16820: - Description:   We are migrating our Kafka cluster from zookeeper to Kraft mode. We are running

[jira] [Created] (KAFKA-16820) Kafka Broker fails to connect to Kraft Controller with no DNS matching

2024-05-22 Thread Arushi Helms (Jira)
Arushi Helms created KAFKA-16820: Summary: Kafka Broker fails to connect to Kraft Controller with no DNS matching Key: KAFKA-16820 URL: https://issues.apache.org/jira/browse/KAFKA-16820 Project: Kafk

Re: [PR] KAFKA-16793: Heartbeat API for upgrading ConsumerGroup [kafka]

2024-05-22 Thread via GitHub
jeffkbkim commented on code in PR #15988: URL: https://github.com/apache/kafka/pull/15988#discussion_r1610462585 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -12401,6 +12436,363 @@ public void testClassicGroupSyncToC

Re: [PR] KAFKA-16793: Heartbeat API for upgrading ConsumerGroup [kafka]

2024-05-22 Thread via GitHub
jeffkbkim commented on code in PR #15988: URL: https://github.com/apache/kafka/pull/15988#discussion_r1610460306 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -12401,6 +12436,363 @@ public void testClassicGroupSyncToC

  1   2   3   >