[jira] [Updated] (KAFKA-15867) Should ConsumerNetworkThread wrap the exception and notify the polling thread?

2023-11-20 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-15867: --- Parent: KAFKA-14246 Issue Type: Sub-task (was: Bug) > Should ConsumerNetworkThread wrap the

[jira] [Created] (KAFKA-15867) Should ConsumerNetworkThread wrap the exception and notify the polling thread?

2023-11-20 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15867: -- Summary: Should ConsumerNetworkThread wrap the exception and notify the polling thread? Key: KAFKA-15867 URL: https://issues.apache.org/jira/browse/KAFKA-15867 Project: K

Re: [PR] KAFKA-15327: ensure the commit manager commit on close [kafka]

2023-11-20 Thread via GitHub
philipnee commented on code in PR #14710: URL: https://github.com/apache/kafka/pull/14710#discussion_r1400134639 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/CommitRequestManagerTest.java: ## @@ -353,7 +380,8 @@ private void testRetriable(final CommitRequ

Re: [PR] KAFKA-15327: ensure the commit manager commit on close [kafka]

2023-11-20 Thread via GitHub
philipnee commented on PR #14710: URL: https://github.com/apache/kafka/pull/14710#issuecomment-1820383565 Hey @lucasbru - Thanks again for taking the time reviewing the PR. I've made some cleanup and changes according to your comments. Let me know if you have further questions. I've

Re: [PR] KAFKA-15327: ensure the commit manager commit on close [kafka]

2023-11-20 Thread via GitHub
philipnee commented on code in PR #14710: URL: https://github.com/apache/kafka/pull/14710#discussion_r1400134639 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/CommitRequestManagerTest.java: ## @@ -353,7 +380,8 @@ private void testRetriable(final CommitRequ

[jira] [Created] (KAFKA-15866) Refactor OffsetFetchRequestState Error handling to be more consistent with OffsetCommitRequestState

2023-11-20 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15866: -- Summary: Refactor OffsetFetchRequestState Error handling to be more consistent with OffsetCommitRequestState Key: KAFKA-15866 URL: https://issues.apache.org/jira/browse/KAFKA-15866

[jira] [Updated] (KAFKA-15866) Refactor OffsetFetchRequestState Error handling to be more consistent with OffsetCommitRequestState

2023-11-20 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-15866: --- Parent: KAFKA-14246 Issue Type: Sub-task (was: Improvement) > Refactor OffsetFetchRequestSt

[jira] [Commented] (KAFKA-15865) Ensure consumer.poll() execute autocommit callback

2023-11-20 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17788298#comment-17788298 ] Philip Nee commented on KAFKA-15865: [~lucasbru] [~cadonna] - Would either of you be

Re: [PR] KAFKA-15327: ensure the commit manager commit on close [kafka]

2023-11-20 Thread via GitHub
philipnee commented on code in PR #14710: URL: https://github.com/apache/kafka/pull/14710#discussion_r1400074015 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerNetworkThread.java: ## @@ -257,10 +257,77 @@ private void closeInternal(final Duration ti

Re: [PR] KAFKA-15327: ensure the commit manager commit on close [kafka]

2023-11-20 Thread via GitHub
philipnee commented on code in PR #14710: URL: https://github.com/apache/kafka/pull/14710#discussion_r1400115792 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -454,6 +491,17 @@ public OffsetFetchRequestState(final Set part

Re: [PR] KAFKA-15327: ensure the commit manager commit on close [kafka]

2023-11-20 Thread via GitHub
philipnee commented on code in PR #14710: URL: https://github.com/apache/kafka/pull/14710#discussion_r1400064245 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CoordinatorRequestManager.java: ## @@ -219,4 +219,14 @@ private void onResponse( public Opti

[jira] [Comment Edited] (KAFKA-15556) Remove NetworkClientDelegate methods isUnavailable, maybeThrowAuthFailure, and tryConnect

2023-11-20 Thread Phuc Hong Tran (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17788233#comment-17788233 ] Phuc Hong Tran edited comment on KAFKA-15556 at 11/21/23 6:23 AM:

Re: [PR] KAFKA-15327: ensure the commit manager commit on close [kafka]

2023-11-20 Thread via GitHub
philipnee commented on code in PR #14710: URL: https://github.com/apache/kafka/pull/14710#discussion_r1400074015 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerNetworkThread.java: ## @@ -257,10 +257,77 @@ private void closeInternal(final Duration ti

[jira] [Updated] (KAFKA-15865) Ensure consumer.poll() execute autocommit callback

2023-11-20 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-15865: --- Labels: consumer-threading-refactor kip-848-preview (was: ) > Ensure consumer.poll() execute autoco

[jira] [Updated] (KAFKA-15865) Ensure consumer.poll() execute autocommit callback

2023-11-20 Thread Philip Nee (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Nee updated KAFKA-15865: --- Parent: KAFKA-14246 Issue Type: Sub-task (was: Bug) > Ensure consumer.poll() execute autoco

Re: [PR] KAFKA-15327: ensure the commit manager commit on close [kafka]

2023-11-20 Thread via GitHub
philipnee commented on code in PR #14710: URL: https://github.com/apache/kafka/pull/14710#discussion_r1400070106 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/events/AutoCommitCompletionBackgroundEvent.java: ## @@ -0,0 +1,23 @@ +/* + * Licensed to the Apac

[jira] [Created] (KAFKA-15865) Ensure consumer.poll() execute autocommit callback

2023-11-20 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15865: -- Summary: Ensure consumer.poll() execute autocommit callback Key: KAFKA-15865 URL: https://issues.apache.org/jira/browse/KAFKA-15865 Project: Kafka Issue Type: Bu

Re: [PR] KAFKA-15327: ensure the commit manager commit on close [kafka]

2023-11-20 Thread via GitHub
philipnee commented on code in PR #14710: URL: https://github.com/apache/kafka/pull/14710#discussion_r1400064652 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -180,12 +188,47 @@ public void maybeAutoCommit(final Map offsets

Re: [PR] KAFKA-15327: ensure the commit manager commit on close [kafka]

2023-11-20 Thread via GitHub
philipnee commented on code in PR #14710: URL: https://github.com/apache/kafka/pull/14710#discussion_r1400064245 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CoordinatorRequestManager.java: ## @@ -219,4 +219,14 @@ private void onResponse( public Opti

Re: [PR] KAFKA-15047: Roll active segment when it breaches the retention policy [kafka]

2023-11-20 Thread via GitHub
satishd commented on code in PR #14766: URL: https://github.com/apache/kafka/pull/14766#discussion_r1399008945 ## core/src/main/scala/kafka/log/UnifiedLog.scala: ## @@ -1424,25 +1425,71 @@ class UnifiedLog(@volatile var logStartOffset: Long, */ private def deleteOldSegmen

[jira] [Updated] (KAFKA-15864) Add more tests asserting the log-start-offset, local-log-start-offset, and HW/LSO/LEO in rolling over segments with tiered storage.

2023-11-20 Thread Satish Duggana (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Duggana updated KAFKA-15864: --- Description: Followup on the [comment|https://github.com/apache/kafka/pull/14766/files#r1395

Re: [PR] KAFKA-15327: ensure the commit manager commit on close [kafka]

2023-11-20 Thread via GitHub
philipnee commented on code in PR #14710: URL: https://github.com/apache/kafka/pull/14710#discussion_r1400055966 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerNetworkThread.java: ## @@ -90,14 +93,13 @@ public void run() { while (runnin

Re: [PR] KAFKA-15778 & KAFKA-15779: Implement metrics manager (KIP-714) [kafka]

2023-11-20 Thread via GitHub
apoorvmittal10 commented on PR #14699: URL: https://github.com/apache/kafka/pull/14699#issuecomment-1820280356 Thanks @junrao for explaining the details. I have updated the PR and removed throttleMs from ClientMetricsManager. I have added a Jira to add respective throttling changes in Quota

[jira] [Created] (KAFKA-15864) Add more tests asserting the log-start-offset, local-log-start-offset, and HW/LSO/LEO in rolling over segments with tiered storage.

2023-11-20 Thread Satish Duggana (Jira)
Satish Duggana created KAFKA-15864: -- Summary: Add more tests asserting the log-start-offset, local-log-start-offset, and HW/LSO/LEO in rolling over segments with tiered storage. Key: KAFKA-15864 URL: https://iss

Re: [PR] KAFKA-14519; [2/N] New coordinator metrics [kafka]

2023-11-20 Thread via GitHub
jolshan merged PR #14387: URL: https://github.com/apache/kafka/pull/14387 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

Re: [PR] KAFKA-14519; [2/N] New coordinator metrics [kafka]

2023-11-20 Thread via GitHub
jolshan commented on PR #14387: URL: https://github.com/apache/kafka/pull/14387#issuecomment-1820273369 Build looks good now. I will merge. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [PR] KAFKA-15778 & KAFKA-15779: Implement metrics manager (KIP-714) [kafka]

2023-11-20 Thread via GitHub
apoorvmittal10 commented on code in PR #14699: URL: https://github.com/apache/kafka/pull/14699#discussion_r1400049044 ## core/src/main/scala/kafka/server/metadata/DynamicConfigPublisher.scala: ## @@ -103,9 +103,16 @@ class DynamicConfigPublisher( ) ca

Re: [PR] KAFKA-15778 & KAFKA-15779: Implement metrics manager (KIP-714) [kafka]

2023-11-20 Thread via GitHub
apoorvmittal10 commented on code in PR #14699: URL: https://github.com/apache/kafka/pull/14699#discussion_r1400048884 ## core/src/main/java/kafka/server/ClientMetricsManager.java: ## @@ -16,31 +16,421 @@ */ package kafka.server; +import kafka.metrics.ClientMetricsConfigs; +

[jira] [Created] (KAFKA-15863) Handle push telemetry throttling with quota manager

2023-11-20 Thread Apoorv Mittal (Jira)
Apoorv Mittal created KAFKA-15863: - Summary: Handle push telemetry throttling with quota manager Key: KAFKA-15863 URL: https://issues.apache.org/jira/browse/KAFKA-15863 Project: Kafka Issue T

Re: [PR] KAFKA-15484: General Rack Aware Assignor [kafka]

2023-11-20 Thread via GitHub
jeffkbkim commented on code in PR #14481: URL: https://github.com/apache/kafka/pull/14481#discussion_r1400044251 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/GeneralUniformAssignmentBuilder.java: ## @@ -14,17 +14,891 @@ * See the License for t

Re: [PR] KAFKA-15778 & KAFKA-15779: Implement metrics manager (KIP-714) [kafka]

2023-11-20 Thread via GitHub
apoorvmittal10 commented on code in PR #14699: URL: https://github.com/apache/kafka/pull/14699#discussion_r1400045702 ## core/src/main/java/kafka/server/ClientMetricsManager.java: ## @@ -16,31 +16,420 @@ */ package kafka.server; +import kafka.metrics.ClientMetricsConfigs;

Re: [PR] KAFKA-15484: General Rack Aware Assignor [kafka]

2023-11-20 Thread via GitHub
jeffkbkim commented on code in PR #14481: URL: https://github.com/apache/kafka/pull/14481#discussion_r1400045359 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/GeneralUniformAssignmentBuilder.java: ## @@ -14,17 +14,891 @@ * See the License for t

Re: [PR] KAFKA-15484: General Rack Aware Assignor [kafka]

2023-11-20 Thread via GitHub
jeffkbkim commented on code in PR #14481: URL: https://github.com/apache/kafka/pull/14481#discussion_r1400044251 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/GeneralUniformAssignmentBuilder.java: ## @@ -14,17 +14,891 @@ * See the License for t

Re: [PR] KAFKA-15586: Clean shutdown detection - server side [kafka]

2023-11-20 Thread via GitHub
CalvinConfluent commented on code in PR #14706: URL: https://github.com/apache/kafka/pull/14706#discussion_r1400042116 ## metadata/src/main/java/org/apache/kafka/controller/BrokersToElrs.java: ## @@ -0,0 +1,161 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under on

Re: [PR] KAFKA-15586: Clean shutdown detection - server side [kafka]

2023-11-20 Thread via GitHub
CalvinConfluent commented on code in PR #14706: URL: https://github.com/apache/kafka/pull/14706#discussion_r1400041537 ## metadata/src/main/java/org/apache/kafka/controller/ClusterControlManager.java: ## @@ -327,10 +333,12 @@ public ControllerResult registerBroker(

Re: [PR] KAFKA-15586: Clean shutdown detection - server side [kafka]

2023-11-20 Thread via GitHub
CalvinConfluent commented on code in PR #14706: URL: https://github.com/apache/kafka/pull/14706#discussion_r1400041854 ## metadata/src/main/java/org/apache/kafka/controller/ReplicationControlManager.java: ## @@ -1772,7 +1814,8 @@ void generateLeaderAndIsrUpdates(String context,

Re: [PR] KAFKA-15586: Clean shutdown detection - server side [kafka]

2023-11-20 Thread via GitHub
CalvinConfluent commented on code in PR #14706: URL: https://github.com/apache/kafka/pull/14706#discussion_r1400041780 ## metadata/src/main/java/org/apache/kafka/controller/BrokersToElrs.java: ## @@ -0,0 +1,161 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under on

[PR] KAFKA-15860: ControllerRegistration must be written out to the metadata image [kafka]

2023-11-20 Thread via GitHub
cmccabe opened a new pull request, #14807: URL: https://github.com/apache/kafka/pull/14807 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-ma

Re: [PR] KAFKA-15586: Clean shutdown detection - server side [kafka]

2023-11-20 Thread via GitHub
CalvinConfluent commented on code in PR #14706: URL: https://github.com/apache/kafka/pull/14706#discussion_r1400036881 ## metadata/src/main/java/org/apache/kafka/controller/PartitionChangeBuilder.java: ## @@ -466,6 +466,10 @@ private void maybeUpdateLastKnownLeader(PartitionCha

Re: [PR] KAFKA-15681: Add support of client-metrics in kafka-configs.sh (KIP-714) [kafka]

2023-11-20 Thread via GitHub
apoorvmittal10 commented on PR #14632: URL: https://github.com/apache/kafka/pull/14632#issuecomment-1820231138 > Seems there is a compilation error on the last run: > > ``` > > [Error] /home/jenkins/jenkins-agent/workspace/Kafka_kafka-pr_PR-14632/core/src/main/scala/kafka/serve

Re: [PR] KAFKA-15585: Add DescribeTopics API server side support [kafka]

2023-11-20 Thread via GitHub
CalvinConfluent commented on code in PR #14612: URL: https://github.com/apache/kafka/pull/14612#discussion_r1400016403 ## core/src/main/scala/kafka/server/KafkaApis.scala: ## @@ -1356,6 +1373,78 @@ class KafkaApis(val requestChannel: RequestChannel, )) } + def handl

Re: [PR] KAFKA-15585: Add DescribeTopics API server side support [kafka]

2023-11-20 Thread via GitHub
CalvinConfluent commented on code in PR #14612: URL: https://github.com/apache/kafka/pull/14612#discussion_r1400016403 ## core/src/main/scala/kafka/server/KafkaApis.scala: ## @@ -1356,6 +1373,78 @@ class KafkaApis(val requestChannel: RequestChannel, )) } + def handl

Re: [PR] KAFKA-15585: Add DescribeTopics API server side support [kafka]

2023-11-20 Thread via GitHub
CalvinConfluent commented on code in PR #14612: URL: https://github.com/apache/kafka/pull/14612#discussion_r1400015238 ## core/src/main/scala/kafka/server/KafkaApis.scala: ## @@ -1356,6 +1373,78 @@ class KafkaApis(val requestChannel: RequestChannel, )) } + def handl

Re: [PR] KAFKA-15585: Add DescribeTopics API server side support [kafka]

2023-11-20 Thread via GitHub
CalvinConfluent commented on code in PR #14612: URL: https://github.com/apache/kafka/pull/14612#discussion_r1400011898 ## clients/src/main/resources/common/message/DescribeTopicPartitionsResponse.json: ## @@ -0,0 +1,66 @@ +// Licensed to the Apache Software Foundation (ASF) unde

Re: [PR] KAFKA-15484: General Rack Aware Assignor [kafka]

2023-11-20 Thread via GitHub
rreddy-22 commented on code in PR #14481: URL: https://github.com/apache/kafka/pull/14481#discussion_r1399989044 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/GeneralUniformAssignmentBuilder.java: ## @@ -14,17 +14,891 @@ * See the License for t

Re: [PR] KAFKA-15484: General Rack Aware Assignor [kafka]

2023-11-20 Thread via GitHub
rreddy-22 commented on code in PR #14481: URL: https://github.com/apache/kafka/pull/14481#discussion_r1399988448 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/GeneralUniformAssignmentBuilder.java: ## @@ -14,17 +14,891 @@ * See the License for t

Re: [PR] KAFKA-15215: [KIP-954] support custom DSL store providers [kafka]

2023-11-20 Thread via GitHub
ableegoldman commented on code in PR #14648: URL: https://github.com/apache/kafka/pull/14648#discussion_r1399973308 ## streams/src/main/java/org/apache/kafka/streams/state/internals/TimestampedKeyValueStoreBuilder.java: ## @@ -91,11 +91,13 @@ private KeyValueStore maybeWrapLoggi

[jira] [Commented] (KAFKA-15556) Remove NetworkClientDelegate methods isUnavailable, maybeThrowAuthFailure, and tryConnect

2023-11-20 Thread Phuc Hong Tran (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17788233#comment-17788233 ] Phuc Hong Tran commented on KAFKA-15556: Hi [~kirktrue], just to clarify, with t

Re: [PR] KAFKA-15215: [KIP-954] support custom DSL store providers [kafka]

2023-11-20 Thread via GitHub
agavra commented on code in PR #14648: URL: https://github.com/apache/kafka/pull/14648#discussion_r1399939069 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/StreamJoinedInternal.java: ## @@ -18,16 +18,30 @@ package org.apache.kafka.streams.kstream.internals

Re: [PR] KAFKA-15585: Add DescribeTopics API server side support [kafka]

2023-11-20 Thread via GitHub
artemlivshits commented on code in PR #14612: URL: https://github.com/apache/kafka/pull/14612#discussion_r1399872558 ## clients/src/main/resources/common/message/DescribeTopicPartitionsResponse.json: ## @@ -0,0 +1,66 @@ +// Licensed to the Apache Software Foundation (ASF) under

Re: [PR] KAFKA-15215: [KIP-954] support custom DSL store providers [kafka]

2023-11-20 Thread via GitHub
agavra commented on code in PR #14648: URL: https://github.com/apache/kafka/pull/14648#discussion_r1399930568 ## streams/src/main/java/org/apache/kafka/streams/state/internals/TimestampedKeyValueStoreBuilder.java: ## @@ -91,11 +91,13 @@ private KeyValueStore maybeWrapLogging(fin

Re: [PR] KAFKA-15681: Add support of client-metrics in kafka-configs.sh (KIP-714) [kafka]

2023-11-20 Thread via GitHub
mjsax commented on PR #14632: URL: https://github.com/apache/kafka/pull/14632#issuecomment-1820056764 Seems there is a compilation error on the last run: ```ask :core:compileScala [Error] /home/jenkins/jenkins-agent/workspace/Kafka_kafka-pr_PR-14632/core/src/main/scala/kafka/server

[jira] [Created] (KAFKA-15862) Remove SecurityManager Support

2023-11-20 Thread Greg Harris (Jira)
Greg Harris created KAFKA-15862: --- Summary: Remove SecurityManager Support Key: KAFKA-15862 URL: https://issues.apache.org/jira/browse/KAFKA-15862 Project: Kafka Issue Type: New Feature

Re: [PR] KAFKA-15362: Resolve offline replicas in metadata cache [kafka]

2023-11-20 Thread via GitHub
cmccabe commented on PR #14737: URL: https://github.com/apache/kafka/pull/14737#issuecomment-1820033474 @soarez : This PR is causing failures in `KRaftClusterTest.testCreateClusterAndPerformReassignment` ``` Gradle Test Run :core:test > Gradle Test Executor 2 > KRaftClusterTest >

[jira] [Created] (KAFKA-15861) In Kraft mode, "ssl.keystore.key" private keys are accesible to all the controllers and brokers

2023-11-20 Thread Jira
Jesús Cea created KAFKA-15861: - Summary: In Kraft mode, "ssl.keystore.key" private keys are accesible to all the controllers and brokers Key: KAFKA-15861 URL: https://issues.apache.org/jira/browse/KAFKA-15861

Re: [PR] MINOR: Always send cumulative failed dirs in HB request [kafka]

2023-11-20 Thread via GitHub
cmccabe merged PR #14770: URL: https://github.com/apache/kafka/pull/14770 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[jira] [Created] (KAFKA-15860) ControllerRegistration must be written out to the metadata image

2023-11-20 Thread Colin McCabe (Jira)
Colin McCabe created KAFKA-15860: Summary: ControllerRegistration must be written out to the metadata image Key: KAFKA-15860 URL: https://issues.apache.org/jira/browse/KAFKA-15860 Project: Kafka

Re: [PR] KAFKA-15778 & KAFKA-15779: Implement metrics manager (KIP-714) [kafka]

2023-11-20 Thread via GitHub
junrao commented on code in PR #14699: URL: https://github.com/apache/kafka/pull/14699#discussion_r1399714279 ## core/src/main/java/kafka/server/ClientMetricsManager.java: ## @@ -16,31 +16,421 @@ */ package kafka.server; +import kafka.metrics.ClientMetricsConfigs; +import k

Re: [PR] MINOR: Mark ConsumerGroupHeartbeat API (v1), OffsetCommit API (v9) and OffsetFetch API (v9) as stable (KIP-848) [kafka]

2023-11-20 Thread via GitHub
jolshan commented on PR #14801: URL: https://github.com/apache/kafka/pull/14801#issuecomment-1819987867 https://github.com/apache/kafka/assets/25566826/3f0c8239-2002-4d66-8455-41f299419738";> Somehow the tests are fixed and existing failures for java 21🤔 (Again, don't think it is your cha

Re: [PR] MINOR: Mark ConsumerGroupHeartbeat API (v1), OffsetCommit API (v9) and OffsetFetch API (v9) as stable (KIP-848) [kafka]

2023-11-20 Thread via GitHub
jolshan commented on PR #14801: URL: https://github.com/apache/kafka/pull/14801#issuecomment-1819985912 The failures here are out of control. I don't think it is related to your change. I will rebuild. -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] KAFKA-15586: Clean shutdown detection - server side [kafka]

2023-11-20 Thread via GitHub
artemlivshits commented on code in PR #14706: URL: https://github.com/apache/kafka/pull/14706#discussion_r1399798771 ## metadata/src/main/java/org/apache/kafka/controller/PartitionChangeBuilder.java: ## @@ -466,6 +466,10 @@ private void maybeUpdateLastKnownLeader(PartitionChang

Re: [PR] [WIP] KAFKA-8677: Flaky test testNoDescribeProduceOrConsumeWithoutTopicDescribeAcl [kafka]

2023-11-20 Thread via GitHub
anatasiavela closed pull request #7267: [WIP] KAFKA-8677: Flaky test testNoDescribeProduceOrConsumeWithoutTopicDescribeAcl URL: https://github.com/apache/kafka/pull/7267 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] KAFKA-15215: [KIP-954] support custom DSL store providers [kafka]

2023-11-20 Thread via GitHub
ableegoldman commented on code in PR #14648: URL: https://github.com/apache/kafka/pull/14648#discussion_r1399822846 ## streams/src/main/java/org/apache/kafka/streams/state/internals/TimestampedKeyValueStoreBuilder.java: ## @@ -91,11 +91,13 @@ private KeyValueStore maybeWrapLoggi

Re: [PR] KAFKA-15215: [KIP-954] support custom DSL store providers [kafka]

2023-11-20 Thread via GitHub
ableegoldman commented on code in PR #14648: URL: https://github.com/apache/kafka/pull/14648#discussion_r1399821160 ## streams/src/main/java/org/apache/kafka/streams/state/DslStoreSuppliers.java: ## @@ -0,0 +1,41 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[PR] MINOR: Increase the delta for a flaky quota test [kafka]

2023-11-20 Thread via GitHub
harinirajendran opened a new pull request, #14806: URL: https://github.com/apache/kafka/pull/14806 Flaky test failing with assertion error just outside the defined threshold. So, bumping up the delta for quota by 1. java.util.concurrent.ExecutionException: org.opentest4j.AssertionFai

Re: [PR] KAFKA-15215: [KIP-954] support custom DSL store providers [kafka]

2023-11-20 Thread via GitHub
ableegoldman commented on code in PR #14648: URL: https://github.com/apache/kafka/pull/14648#discussion_r1399820405 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/StreamJoinedInternal.java: ## @@ -18,16 +18,30 @@ package org.apache.kafka.streams.kstream.int

Re: [PR] KAFKA-15215: [KIP-954] support custom DSL store providers [kafka]

2023-11-20 Thread via GitHub
ableegoldman commented on code in PR #14648: URL: https://github.com/apache/kafka/pull/14648#discussion_r1399812871 ## streams/src/main/java/org/apache/kafka/streams/kstream/internals/OuterStreamJoinStoreFactory.java: ## @@ -0,0 +1,212 @@ +/* + * Licensed to the Apache Software

[PR] MINOR: Increase the delta for a flaky quota test [kafka]

2023-11-20 Thread via GitHub
harinirajendran opened a new pull request, #14805: URL: https://github.com/apache/kafka/pull/14805 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of

Re: [PR] KAFKA-15215: [KIP-954] support custom DSL store providers [kafka]

2023-11-20 Thread via GitHub
ableegoldman commented on PR #14648: URL: https://github.com/apache/kafka/pull/14648#issuecomment-1819878453 Haha yeah it's certainly grown quite a lot over the past few updates -- but for that reason I'm actually fine with keeping it in one PR. The first time I reviewed it there was signif

Re: [PR] KAFKA-15629: Proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery [kafka]

2023-11-20 Thread via GitHub
mjsax commented on code in PR #14570: URL: https://github.com/apache/kafka/pull/14570#discussion_r1399779288 ## streams/src/test/java/org/apache/kafka/streams/integration/IQv2StoreIntegrationTest.java: ## @@ -1632,10 +1729,65 @@ public void shouldHandleRangeQuery(

Re: [PR] KAFKA-15629: Proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery [kafka]

2023-11-20 Thread via GitHub
mjsax commented on code in PR #14570: URL: https://github.com/apache/kafka/pull/14570#discussion_r1399777340 ## streams/src/test/java/org/apache/kafka/streams/integration/IQv2StoreIntegrationTest.java: ## @@ -1584,7 +1646,43 @@ public void shouldHandleKeyQuery( );

Re: [PR] KAFKA-15801: WIP Add hostname and port in NetworkClient logging and increase connection issues logging severity. [kafka]

2023-11-20 Thread via GitHub
GianlucaPrincipini commented on PR #14804: URL: https://github.com/apache/kafka/pull/14804#issuecomment-1819828622 Any suggestions to retrieve port in the best possible way? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[PR] [KAFKA-15801][WIP] Add hostname and port in netword NetworkClient logging and increase connection issues severity [kafka]

2023-11-20 Thread via GitHub
GianlucaPrincipini opened a new pull request, #14804: URL: https://github.com/apache/kafka/pull/14804 ## Description When a component of the Kafka broker tries to reach another broker within the cluster the logging should be more elaborate and include the IP/hostname and port it tries to

Re: [PR] KAFKA-15143: Adding in MockFixedKeyProcessorContext and Test [kafka]

2023-11-20 Thread via GitHub
lihaosky commented on code in PR #14605: URL: https://github.com/apache/kafka/pull/14605#discussion_r1399755568 ## streams/test-utils/src/test/java/org/apache/kafka/streams/test/MockFixedKeyProcessorContextTest.java: ## @@ -0,0 +1,315 @@ +/* + * Licensed to the Apache Software F

[PR] Update Streams API broker compat table [kafka]

2023-11-20 Thread via GitHub
JimGalasyn opened a new pull request, #14803: URL: https://github.com/apache/kafka/pull/14803 Update the table for v3.6. Also needs to be cherry-picked to the 3.6 branch. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] KAFKA-15618: Kafka metrics collector and supporting classes (KIP-714) [kafka]

2023-11-20 Thread via GitHub
xvrl commented on code in PR #14620: URL: https://github.com/apache/kafka/pull/14620#discussion_r1399702579 ## clients/src/main/java/org/apache/kafka/common/telemetry/internals/KafkaMetricsCollector.java: ## @@ -0,0 +1,334 @@ +/* + * Licensed to the Apache Software Foundation (A

Re: [PR] KAFKA-15363: Broker log directory failure changes [kafka]

2023-11-20 Thread via GitHub
OmniaGM commented on code in PR #14790: URL: https://github.com/apache/kafka/pull/14790#discussion_r1399670915 ## core/src/main/scala/kafka/server/ReplicaAlterLogDirsThread.scala: ## @@ -76,13 +82,49 @@ class ReplicaAlterLogDirsThread(name: String, futureLog.updateHighWater

Re: [PR] KAFKA-15363: Broker log directory failure changes [kafka]

2023-11-20 Thread via GitHub
OmniaGM commented on code in PR #14790: URL: https://github.com/apache/kafka/pull/14790#discussion_r1399669761 ## core/src/main/scala/kafka/server/ReplicaAlterLogDirsThread.scala: ## @@ -76,13 +82,49 @@ class ReplicaAlterLogDirsThread(name: String, futureLog.updateHighWater

Re: [PR] KAFKA-15363: Broker log directory failure changes [kafka]

2023-11-20 Thread via GitHub
OmniaGM commented on code in PR #14790: URL: https://github.com/apache/kafka/pull/14790#discussion_r1399667766 ## core/src/main/scala/kafka/server/ReplicaAlterLogDirsThread.scala: ## @@ -76,13 +82,49 @@ class ReplicaAlterLogDirsThread(name: String, futureLog.updateHighWater

Re: [PR] KAFKA-15816: Fix leaked sockets in streams tests [kafka]

2023-11-20 Thread via GitHub
gharris1727 commented on code in PR #14769: URL: https://github.com/apache/kafka/pull/14769#discussion_r1399656534 ## streams/src/test/java/org/apache/kafka/streams/integration/IQv2IntegrationTest.java: ## @@ -418,6 +418,8 @@ public String metricsScope() { })

Re: [PR] KAFKA-15629: Proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery [kafka]

2023-11-20 Thread via GitHub
hanyuzheng7 commented on code in PR #14570: URL: https://github.com/apache/kafka/pull/14570#discussion_r1399656268 ## streams/src/test/java/org/apache/kafka/streams/integration/IQv2StoreIntegrationTest.java: ## @@ -1632,10 +1729,65 @@ public void shouldHandleRangeQuery(

Re: [PR] KAFKA-15629: Proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery [kafka]

2023-11-20 Thread via GitHub
hanyuzheng7 commented on code in PR #14570: URL: https://github.com/apache/kafka/pull/14570#discussion_r1399652671 ## streams/src/test/java/org/apache/kafka/streams/integration/IQv2StoreIntegrationTest.java: ## @@ -1584,7 +1646,43 @@ public void shouldHandleKeyQuery( )

Re: [PR] KAFKA-15629: Proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery [kafka]

2023-11-20 Thread via GitHub
hanyuzheng7 commented on code in PR #14570: URL: https://github.com/apache/kafka/pull/14570#discussion_r1399632055 ## streams/src/test/java/org/apache/kafka/streams/integration/IQv2StoreIntegrationTest.java: ## @@ -1584,7 +1646,43 @@ public void shouldHandleKeyQuery( )

Re: [PR] KAFKA-15629: Proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery [kafka]

2023-11-20 Thread via GitHub
hanyuzheng7 commented on code in PR #14570: URL: https://github.com/apache/kafka/pull/14570#discussion_r1399632055 ## streams/src/test/java/org/apache/kafka/streams/integration/IQv2StoreIntegrationTest.java: ## @@ -1584,7 +1646,43 @@ public void shouldHandleKeyQuery( )

Re: [PR] KAFKA-15363: Broker log directory failure changes [kafka]

2023-11-20 Thread via GitHub
OmniaGM commented on code in PR #14790: URL: https://github.com/apache/kafka/pull/14790#discussion_r1399627999 ## core/src/main/scala/kafka/server/ReplicaAlterLogDirsThread.scala: ## @@ -76,13 +82,49 @@ class ReplicaAlterLogDirsThread(name: String, futureLog.updateHighWater

Re: [PR] KAFKA-15363: Broker log directory failure changes [kafka]

2023-11-20 Thread via GitHub
OmniaGM commented on code in PR #14790: URL: https://github.com/apache/kafka/pull/14790#discussion_r1399627137 ## core/src/main/scala/kafka/server/ReplicaAlterLogDirsThread.scala: ## @@ -76,13 +82,49 @@ class ReplicaAlterLogDirsThread(name: String, futureLog.updateHighWater

Re: [PR] KAFKA-15629: Proposal to introduce IQv2 Query Types: TimestampedKeyQuery and TimestampedRangeQuery [kafka]

2023-11-20 Thread via GitHub
mjsax commented on code in PR #14570: URL: https://github.com/apache/kafka/pull/14570#discussion_r1399617212 ## streams/src/test/java/org/apache/kafka/streams/integration/IQv2StoreIntegrationTest.java: ## @@ -1584,7 +1646,43 @@ public void shouldHandleKeyQuery( );

Re: [PR] KAFKA-15363: Broker log directory failure changes [kafka]

2023-11-20 Thread via GitHub
OmniaGM commented on code in PR #14790: URL: https://github.com/apache/kafka/pull/14790#discussion_r1399605671 ## core/src/main/scala/kafka/server/ReplicaAlterLogDirsThread.scala: ## @@ -76,13 +82,49 @@ class ReplicaAlterLogDirsThread(name: String, futureLog.updateHighWater

Re: [PR] MINOR: Small LogValidator clean ups [kafka]

2023-11-20 Thread via GitHub
hachikuji merged PR #14697: URL: https://github.com/apache/kafka/pull/14697 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apach

Re: [PR] MINOR: Small LogValidator clean ups [kafka]

2023-11-20 Thread via GitHub
hachikuji commented on PR #14697: URL: https://github.com/apache/kafka/pull/14697#issuecomment-1819609057 @junrao I triggered a few builds and didn't see much consistency in the test failures. I tried some of the tests locally and they passed. I will go ahead and merge to trunk. -- This

Re: [PR] KAFKA-15417 flip joinSpuriousLookBackTimeMs and emit non-joined items [kafka]

2023-11-20 Thread via GitHub
lihaosky commented on code in PR #14426: URL: https://github.com/apache/kafka/pull/14426#discussion_r1397782005 ## streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamKStreamLeftJoinTest.java: ## @@ -609,17 +842,18 @@ public void testOrdering() {

Re: [PR] KAFKA-15586: Clean shutdown detection - server side [kafka]

2023-11-20 Thread via GitHub
CalvinConfluent commented on code in PR #14706: URL: https://github.com/apache/kafka/pull/14706#discussion_r1399579994 ## metadata/src/main/java/org/apache/kafka/controller/ClusterControlManager.java: ## @@ -327,10 +333,12 @@ public ControllerResult registerBroker(

Re: [PR] KAFKA-15586: Clean shutdown detection - server side [kafka]

2023-11-20 Thread via GitHub
CalvinConfluent commented on code in PR #14706: URL: https://github.com/apache/kafka/pull/14706#discussion_r1399575912 ## metadata/src/main/java/org/apache/kafka/controller/ReplicationControlManager.java: ## @@ -1294,8 +1315,11 @@ void handleBrokerFenced(int brokerId, List reco

Re: [PR] KAFKA-15022: Detect negative cycle from one source [kafka]

2023-11-20 Thread via GitHub
lihaosky commented on code in PR #14696: URL: https://github.com/apache/kafka/pull/14696#discussion_r1399571144 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/assignment/Graph.java: ## @@ -117,6 +131,8 @@ private Graph(final boolean isResidualGraph) {

Re: [PR] KAFKA-15705: Add integration tests for Heartbeat API and GroupLeave API [kafka]

2023-11-20 Thread via GitHub
dongnuo123 commented on code in PR #14656: URL: https://github.com/apache/kafka/pull/14656#discussion_r1399570354 ## core/src/test/scala/unit/kafka/server/HeartbeatRequestTest.scala: ## @@ -0,0 +1,199 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more

Re: [PR] KAFKA-15363: Broker log directory failure changes [kafka]

2023-11-20 Thread via GitHub
OmniaGM commented on code in PR #14790: URL: https://github.com/apache/kafka/pull/14790#discussion_r1399564930 ## core/src/main/java/kafka/server/AssignmentsManager.java: ## @@ -210,6 +220,9 @@ public void run() throws Exception { channelManager.sendRequest(new Assi

Re: [PR] KAFKA-15417 flip joinSpuriousLookBackTimeMs and emit non-joined items [kafka]

2023-11-20 Thread via GitHub
lihaosky commented on code in PR #14426: URL: https://github.com/apache/kafka/pull/14426#discussion_r1399561823 ## streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamKStreamOuterJoinTest.java: ## @@ -791,7 +791,7 @@ public void shouldNotEmitLeftJoinResultFo

Re: [PR] KAFKA-15363: Broker log directory failure changes [kafka]

2023-11-20 Thread via GitHub
OmniaGM commented on code in PR #14790: URL: https://github.com/apache/kafka/pull/14790#discussion_r1399559787 ## core/src/main/java/kafka/server/AssignmentsManager.java: ## @@ -146,6 +153,9 @@ public void run() throws Exception { log.debug("Received new assignm

Re: [PR] KAFKA-15363: Broker log directory failure changes [kafka]

2023-11-20 Thread via GitHub
OmniaGM commented on code in PR #14790: URL: https://github.com/apache/kafka/pull/14790#discussion_r1399558750 ## core/src/main/java/kafka/server/AssignmentsManager.java: ## @@ -146,6 +153,9 @@ public void run() throws Exception { log.debug("Received new assignm

Re: [PR] KAFKA-15681: Add support of client-metrics in kafka-configs.sh (KIP-714) [kafka]

2023-11-20 Thread via GitHub
junrao commented on PR #14632: URL: https://github.com/apache/kafka/pull/14632#issuecomment-1819538489 @apoorvmittal10 : Thanks for triaging the failed tests. There is still no green build though. -- This is an automated message from the Apache Git Service. To respond to the message, plea

  1   2   >