Re: [PR] KAFKA-15673: Adding client metrics resource types (KIP-714) [kafka]

2023-10-25 Thread via GitHub
apoorvmittal10 commented on code in PR #14621: URL: https://github.com/apache/kafka/pull/14621#discussion_r1372047546 ## core/src/main/scala/kafka/server/ZkConfigManager.scala: ## @@ -38,7 +38,8 @@ object ConfigType { val User = "users" val Broker = "brokers" val Ip =

Re: [PR] KAFKA-15673: Adding client metrics resource types (KIP-714) [kafka]

2023-10-25 Thread via GitHub
apoorvmittal10 commented on code in PR #14621: URL: https://github.com/apache/kafka/pull/14621#discussion_r1372046033 ## core/src/main/scala/kafka/metrics/ClientMetricsMetadata.scala: ## @@ -0,0 +1,130 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or

Re: [PR] KAFKA-15673: Adding client metrics resource types (KIP-714) [kafka]

2023-10-25 Thread via GitHub
apoorvmittal10 commented on code in PR #14621: URL: https://github.com/apache/kafka/pull/14621#discussion_r1372045362 ## core/src/main/scala/kafka/metrics/ClientMetricsConfig.scala: ## @@ -0,0 +1,141 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more

[jira] [Commented] (KAFKA-15564) Kafka 3.5.1- Mirror Maker 2 replicating the wrong consumer group offsets in destination

2023-10-25 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17779568#comment-17779568 ] Greg Harris commented on KAFKA-15564: - [~hemanthsavasere] Were you able to work around this problem

Re: [PR] KAFKA-15673: Adding client metrics resource types (KIP-714) [kafka]

2023-10-25 Thread via GitHub
apoorvmittal10 commented on code in PR #14621: URL: https://github.com/apache/kafka/pull/14621#discussion_r1372044581 ## clients/src/main/java/org/apache/kafka/clients/admin/KafkaAdminClient.java: ## @@ -2619,6 +2619,9 @@ private ConfigEntry.ConfigSource

Re: [PR] MINOR: avoid blocking for randomness in DefaultRecordBatchTest [kafka]

2023-10-25 Thread via GitHub
divijvaidya commented on code in PR #14625: URL: https://github.com/apache/kafka/pull/14625#discussion_r1372035225 ## clients/src/test/java/org/apache/kafka/common/record/DefaultRecordBatchTest.java: ## @@ -63,15 +61,9 @@ import static org.mockito.Mockito.when; public class

Re: [PR] Kafka 15680 [kafka]

2023-10-25 Thread via GitHub
C0urante commented on PR #14630: URL: https://github.com/apache/kafka/pull/14630#issuecomment-1779644800 @kumarpritam863 is this the same as https://github.com/apache/kafka/pull/12622 and https://issues.apache.org/jira/browse/KAFKA-14220? -- This is an automated message from the Apache

Re: [PR] KAFKA-15582: Move the clean shutdown file to the storage package [kafka]

2023-10-25 Thread via GitHub
junrao commented on code in PR #14603: URL: https://github.com/apache/kafka/pull/14603#discussion_r1372017036 ## storage/src/main/java/org/apache/kafka/storage/internals/checkpoint/CleanShutdownFileHandler.java: ## @@ -73,10 +77,8 @@ void write(long brokerEpoch, int version)

Re: [PR] KAFKA-15633: Fix overwrite of meta.properties at startup to handle JBOD. [kafka]

2023-10-25 Thread via GitHub
pprovenzano closed pull request #14578: KAFKA-15633: Fix overwrite of meta.properties at startup to handle JBOD. URL: https://github.com/apache/kafka/pull/14578 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] KAFKA-15633: Fix overwrite of meta.properties at startup to handle JBOD. [kafka]

2023-10-25 Thread via GitHub
pprovenzano commented on PR #14578: URL: https://github.com/apache/kafka/pull/14578#issuecomment-1779628602 Closing this PR in favor of https://github.com/apache/kafka/pull/14628 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] KAFKA-15627: KIP-951's Leader discovery optimisations on the client [kafka]

2023-10-25 Thread via GitHub
msn-tldr commented on PR #14564: URL: https://github.com/apache/kafka/pull/14564#issuecomment-1779551983 @AndrewJSchofield & @kirktrue addressed the feedback so far! thanks for the reviews. -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] MINOR: LeaveGroupResponse v0 - v2 loses its member under certain error conditions [kafka]

2023-10-25 Thread via GitHub
wolfchimneyrock commented on PR #14635: URL: https://github.com/apache/kafka/pull/14635#issuecomment-1779515287 Here is a broker trace: ``` [2023-10-24 01:17:17,214] ERROR [KafkaApi-28598] Unexpected error handling request RequestHeader(apiKey=LEAVE_GROUP, apiVersion=1,

[jira] [Created] (KAFKA-15684) Add support to describe all subscriptions through utility

2023-10-25 Thread Apoorv Mittal (Jira)
Apoorv Mittal created KAFKA-15684: - Summary: Add support to describe all subscriptions through utility Key: KAFKA-15684 URL: https://issues.apache.org/jira/browse/KAFKA-15684 Project: Kafka

[jira] [Created] (KAFKA-15683) Delete subscription from metadata when all configs are deleted

2023-10-25 Thread Apoorv Mittal (Jira)
Apoorv Mittal created KAFKA-15683: - Summary: Delete subscription from metadata when all configs are deleted Key: KAFKA-15683 URL: https://issues.apache.org/jira/browse/KAFKA-15683 Project: Kafka

[PR] MINOR: Fix UT stubbing [kafka]

2023-10-25 Thread via GitHub
iamlockon opened a new pull request, #14637: URL: https://github.com/apache/kafka/pull/14637 *More detailed description of your change, Originally the assert and stub statements are not verifying correctly, so `assertEquals` will always be true. I have checked the whole file and fixed

Re: [PR] MINOR: LeaveGroupResponse v0 - v2 loses its member under certain error conditions [kafka]

2023-10-25 Thread via GitHub
wolfchimneyrock commented on PR #14635: URL: https://github.com/apache/kafka/pull/14635#issuecomment-1779439685 I'm not sure I can elaborate on what the underlying error response that would have been sent with the LeaveGroup is, since the current code raises that exception without

[jira] [Commented] (KAFKA-15200) verify pre-requisite at start of release.py

2023-10-25 Thread Owen C.H. Leung (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17779537#comment-17779537 ] Owen C.H. Leung commented on KAFKA-15200: - [~divijvaidya] I've created PR for this issue. Can I

[PR] Add pre-requisite check in release.py [kafka]

2023-10-25 Thread via GitHub
Owen-CH-Leung opened a new pull request, #14636: URL: https://github.com/apache/kafka/pull/14636 As per the request in [KAFKA-15200](https://issues.apache.org/jira/browse/KAFKA-15200), This PR adds pre-requitsite-check in the `release.py` before the release process begins ###

Re: [PR] MINOR: LeaveGroupResponse v0 - v2 loses its member under certain error conditions [kafka]

2023-10-25 Thread via GitHub
wolfchimneyrock commented on PR #14635: URL: https://github.com/apache/kafka/pull/14635#issuecomment-1779425711 > @wolfchimneyrock Thanks for the PR. I am not to fully understand the issue yet. Could you please elaborate a bit more about the condition leading to it? there is more

Re: [PR] MINOR: LeaveGroupResponse v0 - v2 loses its member under certain error conditions [kafka]

2023-10-25 Thread via GitHub
dajac commented on PR #14635: URL: https://github.com/apache/kafka/pull/14635#issuecomment-1779417281 @wolfchimneyrock Thanks for the PR. I am not to fully understand the issue yet. Could you please elaborate a bit more about the condition leading to it? -- This is an automated message

Re: [PR] MINOR: Rewrite the meta.properties handling code in Java and fix some issues [kafka]

2023-10-25 Thread via GitHub
soarez commented on code in PR #14628: URL: https://github.com/apache/kafka/pull/14628#discussion_r1371749943 ## metadata/src/main/java/org/apache/kafka/metadata/properties/MetaProperties.java: ## @@ -0,0 +1,266 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

Re: [PR] KAFKA-14133: Replace Easymock with Mockito in StreamsProducerTest, TopologyMetadataTest & GlobalStateStoreProviderTest [kafka]

2023-10-25 Thread via GitHub
divijvaidya commented on PR #14410: URL: https://github.com/apache/kafka/pull/14410#issuecomment-1779407230 > @divijvaidya Do you have cycles to review this? Sure, I can get to this before the end this week. -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] KAFKA-15355: Message schema changes [kafka]

2023-10-25 Thread via GitHub
rondagostino commented on code in PR #14290: URL: https://github.com/apache/kafka/pull/14290#discussion_r1371865719 ## metadata/src/main/java/org/apache/kafka/controller/PartitionChangeBuilder.java: ## @@ -54,6 +54,7 @@ public static boolean

Re: [PR] KAFKA-15598 & KAFKA-15461: Add integration tests for DescribeGroups API, DeleteGroups API, OffsetDelete API and ListGroups API [kafka]

2023-10-25 Thread via GitHub
dajac commented on code in PR #14537: URL: https://github.com/apache/kafka/pull/14537#discussion_r1371838869 ## core/src/test/scala/unit/kafka/server/DescribeGroupsRequestTest.scala: ## @@ -0,0 +1,118 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or

Re: [PR] KAFKA-15046: Get rid of unnecessary fsyncs inside UnifiedLog.lock to stabilize performance [kafka]

2023-10-25 Thread via GitHub
divijvaidya commented on PR #14242: URL: https://github.com/apache/kafka/pull/14242#issuecomment-1779404361 > If we ignore producer-state-flush failure here, recovery-point might be incremented even with stale on-disk producer state snapshot. So, in case of restart after power failure, the

[PR] MINOR: LeaveGroupResponse v0 - v2 loses its member under certain error conditions [kafka]

2023-10-25 Thread via GitHub
wolfchimneyrock opened a new pull request, #14635: URL: https://github.com/apache/kafka/pull/14635 KIP-848 introduced this check, but we have seen since upgrading to broker 3.4 with both Sarama and Librdkafka clients when issuing LeaveGroup and getting an error, the single member gets lost

Re: [PR] KAFKA-15647: Fix the different behavior in error handling between the old and new group coordinator [kafka]

2023-10-25 Thread via GitHub
dajac commented on code in PR #14589: URL: https://github.com/apache/kafka/pull/14589#discussion_r1371818152 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinatorService.java: ## @@ -310,16 +313,22 @@ public CompletableFuture joinGroup(

Re: [PR] KAFKA-15355: Message schema changes [kafka]

2023-10-25 Thread via GitHub
rondagostino commented on code in PR #14290: URL: https://github.com/apache/kafka/pull/14290#discussion_r1371825053 ## metadata/src/main/resources/common/metadata/PartitionRecord.json: ## @@ -47,6 +47,13 @@ "about": "The eligible leader replicas of this partition." },

[PR] KAFKA-15348: add support for 'range IQs with versioned state stores (KIP-969) [kafka]

2023-10-25 Thread via GitHub
aliehsaeedii opened a new pull request, #14634: URL: https://github.com/apache/kafka/pull/14634 This is the PoC PR for KIP-969. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] KAFKA-14133: Replace Easymock with Mockito in StreamsProducerTest, TopologyMetadataTest & GlobalStateStoreProviderTest [kafka]

2023-10-25 Thread via GitHub
ijuma commented on code in PR #14410: URL: https://github.com/apache/kafka/pull/14410#discussion_r1371770193 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/StreamsProducerTest.java: ## @@ -197,8 +203,7 @@ public void before() { );

Re: [PR] KAFKA-14133: Replace Easymock with Mockito in StreamsProducerTest, TopologyMetadataTest & GlobalStateStoreProviderTest [kafka]

2023-10-25 Thread via GitHub
ijuma commented on PR #14410: URL: https://github.com/apache/kafka/pull/14410#issuecomment-1779286800 @divijvaidya Do you have cycles to review this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] KAFKA-14132; Replace EasyMock with Mockito in StandaloneHerderTest [kafka]

2023-10-25 Thread via GitHub
ijuma commented on PR #12728: URL: https://github.com/apache/kafka/pull/12728#issuecomment-1779282256 @mimaison Do you have the cycles to review this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] KAFKA-15681: Add support of client-metrics in kafka-configs.sh (KIP-714) [kafka]

2023-10-25 Thread via GitHub
apoorvmittal10 commented on PR #14632: URL: https://github.com/apache/kafka/pull/14632#issuecomment-1779271622 Build depends on PR - https://github.com/apache/kafka/pull/14621 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] KAFKA-15670: add "inter.broker.listener.name" config in KRaft controller config [kafka]

2023-10-25 Thread via GitHub
ppatierno commented on code in PR #14631: URL: https://github.com/apache/kafka/pull/14631#discussion_r1371736949 ## docs/ops.html: ## @@ -3822,6 +3822,9 @@ Provisioning the KRaft controller quorum # The inter broker listener in brokers to allow KRaft controller send RPCs to

[jira] [Updated] (KAFKA-15682) Ensure internal remote log metadata topic does not expire its segments before deleting user-topic segments

2023-10-25 Thread Kamal Chandraprakash (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamal Chandraprakash updated KAFKA-15682: - Description: One of the implementation of RemoteLogMetadataManager is

Re: [PR] KAFKA-15670: add "inter.broker.listener.name" config in KRaft controller config [kafka]

2023-10-25 Thread via GitHub
showuon commented on code in PR #14631: URL: https://github.com/apache/kafka/pull/14631#discussion_r1371720987 ## docs/ops.html: ## @@ -3819,6 +3819,9 @@ Provisioning the KRaft controller quorum # ZooKeeper client configuration zookeeper.connect=localhost:2181 +# The inter

Re: [PR] KAFKA-15614: Define interfaces and classes for client telemetry [kafka]

2023-10-25 Thread via GitHub
apoorvmittal10 commented on code in PR #14575: URL: https://github.com/apache/kafka/pull/14575#discussion_r1371720108 ## clients/src/main/java/org/apache/kafka/common/telemetry/ClientTelemetry.java: ## @@ -0,0 +1,44 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[jira] [Updated] (KAFKA-15682) Ensure internal remote log metadata topic does not expire its segments before deleting user-topic segments

2023-10-25 Thread Kamal Chandraprakash (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamal Chandraprakash updated KAFKA-15682: - Description: One of the implementation of RemoteLogMetadataManager is

Re: [PR] KAFKA-15632: Drop the invalid remote log metadata events [kafka]

2023-10-25 Thread via GitHub
kamalcph commented on code in PR #14576: URL: https://github.com/apache/kafka/pull/14576#discussion_r1371713733 ## storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/RemoteLogMetadataCache.java: ## @@ -157,13 +157,20 @@ public void

[jira] [Updated] (KAFKA-15682) Ensure internal remote log metadata topic does not expire its segments before deleting user-topic segments

2023-10-25 Thread Kamal Chandraprakash (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamal Chandraprakash updated KAFKA-15682: - Description: One of the implementation of RemoteLogMetadataManager is

[jira] [Updated] (KAFKA-15682) Ensure internal remote log metadata topic does not expire its segments before deleting user-topic segments

2023-10-25 Thread Kamal Chandraprakash (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamal Chandraprakash updated KAFKA-15682: - Description: One of the implementation of RemoteLogMetadataManager is

[jira] [Created] (KAFKA-15682) Ensure internal remote log metadata topic does not expire its segments before deleting user-topic segments

2023-10-25 Thread Kamal Chandraprakash (Jira)
Kamal Chandraprakash created KAFKA-15682: Summary: Ensure internal remote log metadata topic does not expire its segments before deleting user-topic segments Key: KAFKA-15682 URL:

[PR] Add explicit exit call to each shell command [kafka]

2023-10-25 Thread via GitHub
tkornai opened a new pull request, #14633: URL: https://github.com/apache/kafka/pull/14633 Using a custom security.provider may keep shell commands running indefinitely if System.exit() is not called explicitly. This behaviour is due to the reason that a security.provider implementation

Re: [PR] KAFKA-14132; Replace EasyMock with Mockito in StandaloneHerderTest [kafka]

2023-10-25 Thread via GitHub
mdedetrich commented on PR #12728: URL: https://github.com/apache/kafka/pull/12728#issuecomment-1779161332 @mimaison @ijuma I have just finished PR and I can confirm the tests are passing locally. Due to the fact we are no longer dealing with static classes, the setting up of the static

Re: [PR] KAFKA-15633: Fix overwrite of meta.properties at startup to handle JBOD. [kafka]

2023-10-25 Thread via GitHub
pprovenzano commented on code in PR #14578: URL: https://github.com/apache/kafka/pull/14578#discussion_r1371668063 ## core/src/main/scala/kafka/log/LogManager.scala: ## @@ -270,26 +270,38 @@ class LogManager(logDirs: Seq[File], def directoryId(dir: String): Option[Uuid] =

Re: [PR] KAFKA-15681: Add support of client-metrics in kafka-configs.sh (KIP-714) [kafka]

2023-10-25 Thread via GitHub
apoorvmittal10 commented on code in PR #14632: URL: https://github.com/apache/kafka/pull/14632#discussion_r1371642341 ## core/src/main/scala/kafka/admin/ConfigCommand.scala: ## @@ -536,6 +552,8 @@ object ConfigCommand extends Logging { adminClient.listTopics(new

Re: [PR] KAFKA-15681: Add support of client-metrics in kafka-configs.sh (KIP-714) [kafka]

2023-10-25 Thread via GitHub
apoorvmittal10 commented on code in PR #14632: URL: https://github.com/apache/kafka/pull/14632#discussion_r1371639259 ## core/src/main/scala/kafka/admin/ConfigCommand.scala: ## @@ -576,6 +594,8 @@ object ConfigCommand extends Logging { if (entityName.nonEmpty)

Re: [PR] KAFKA-15681: Add support of client-metrics in kafka-configs.sh (KIP-714) [kafka]

2023-10-25 Thread via GitHub
apoorvmittal10 commented on code in PR #14632: URL: https://github.com/apache/kafka/pull/14632#discussion_r1371629188 ## core/src/main/scala/kafka/admin/ConfigCommand.scala: ## @@ -445,6 +446,21 @@ object ConfigCommand extends Logging { if (unknownConfigs.nonEmpty)

Re: [PR] KAFKA-15046: Get rid of unnecessary fsyncs inside UnifiedLog.lock to stabilize performance [kafka]

2023-10-25 Thread via GitHub
ocadaruma commented on PR #14242: URL: https://github.com/apache/kafka/pull/14242#issuecomment-1779086343 @divijvaidya Thank you for your review. > But now, since we are flushing snapshot async & quietly During reading your comment, I realized that "quietly" could be a problem

Re: [PR] KAFKA-14517:Implement regex subscriptions [kafka]

2023-10-25 Thread via GitHub
vamossagar12 commented on code in PR #14327: URL: https://github.com/apache/kafka/pull/14327#discussion_r1371564208 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroup.java: ## @@ -727,20 +728,42 @@ private void

Re: [PR] KAFKA-15046: Get rid of unnecessary fsyncs inside UnifiedLog.lock to stabilize performance [kafka]

2023-10-25 Thread via GitHub
ocadaruma commented on PR #14242: URL: https://github.com/apache/kafka/pull/14242#issuecomment-1779076146 @divijvaidya Thank you for your review. > > But now, since we are flushing snapshot async & quietly During reading your comment, I found that "quietly" -- This is an

Re: [PR] KAFKA-15633: Fix overwrite of meta.properties at startup to handle JBOD. [kafka]

2023-10-25 Thread via GitHub
soarez commented on code in PR #14578: URL: https://github.com/apache/kafka/pull/14578#discussion_r1371577705 ## core/src/main/scala/kafka/log/LogManager.scala: ## @@ -270,26 +270,38 @@ class LogManager(logDirs: Seq[File], def directoryId(dir: String): Option[Uuid] =

Re: [PR] KAFKA-15681: Add support of client-metrics in kafka-configs.sh (KIP-714) [kafka]

2023-10-25 Thread via GitHub
AndrewJSchofield commented on code in PR #14632: URL: https://github.com/apache/kafka/pull/14632#discussion_r1371581178 ## core/src/main/scala/kafka/admin/ConfigCommand.scala: ## @@ -445,6 +446,21 @@ object ConfigCommand extends Logging { if (unknownConfigs.nonEmpty)

[jira] [Assigned] (KAFKA-15363) Broker log directory failure changes

2023-10-25 Thread Omnia Ibrahim (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Omnia Ibrahim reassigned KAFKA-15363: - Assignee: Omnia Ibrahim > Broker log directory failure changes >

[jira] [Assigned] (KAFKA-15364) Handle log directory failure in the Controller

2023-10-25 Thread Omnia Ibrahim (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Omnia Ibrahim reassigned KAFKA-15364: - Assignee: (was: Omnia Ibrahim) > Handle log directory failure in the Controller >

Re: [PR] KAFKA-15514: Metadata records Replicas->Assignment [kafka]

2023-10-25 Thread via GitHub
soarez commented on PR #14516: URL: https://github.com/apache/kafka/pull/14516#issuecomment-1779020910 Thanks for the review and comments @OmniaGM . Depending on the outcome of the current discussion in #14290 there may be some bigger changes to this PR, so I'll wait until that's resolved

Re: [PR] KAFKA-15670: add "inter.broker.listener.name" config in KRaft controller config [kafka]

2023-10-25 Thread via GitHub
ppatierno commented on code in PR #14631: URL: https://github.com/apache/kafka/pull/14631#discussion_r1371542283 ## docs/ops.html: ## @@ -3819,6 +3819,9 @@ Provisioning the KRaft controller quorum # ZooKeeper client configuration zookeeper.connect=localhost:2181 +# The

Re: [PR] KAFKA-15681: Add support of client-metrics in kafka-configs.sh (KIP-714) [kafka]

2023-10-25 Thread via GitHub
AndrewJSchofield commented on code in PR #14632: URL: https://github.com/apache/kafka/pull/14632#discussion_r1371529124 ## core/src/main/scala/kafka/admin/ConfigCommand.scala: ## @@ -445,6 +446,21 @@ object ConfigCommand extends Logging { if (unknownConfigs.nonEmpty)

[PR] KAFKA-15681: Add support of client-metrics in kafka-configs.sh (KIP-714) [kafka]

2023-10-25 Thread via GitHub
apoorvmittal10 opened a new pull request, #14632: URL: https://github.com/apache/kafka/pull/14632 The PR adds support of alter/describe configs for client-metrics as defined in

Re: [PR] KAFKA-15670: add "inter.broker.listener.name" config in KRaft controller config [kafka]

2023-10-25 Thread via GitHub
showuon commented on PR #14631: URL: https://github.com/apache/kafka/pull/14631#issuecomment-1778977394 @mumrah @cmccabe , call for review. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[PR] KAFKA-15670: add "inter.broker.listener.name" config in KRaft controller config [kafka]

2023-10-25 Thread via GitHub
showuon opened a new pull request, #14631: URL: https://github.com/apache/kafka/pull/14631 During ZK migrating to KRaft, before entering dual-write mode, the KRaft controller will send RPCs (i.e. UpdateMetadataRequest, LeaderAndIsrRequest, and StopReplicaRequest) to the brokers. Currently,

[jira] [Updated] (KAFKA-15670) KRaft controller should set inter broker listener when migration

2023-10-25 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen updated KAFKA-15670: -- Description: During ZK migrating to KRaft, before entering dual-write mode, the KRaft controller

Re: [PR] KAFKA-15046: Get rid of unnecessary fsyncs inside UnifiedLog.lock to stabilize performance [kafka]

2023-10-25 Thread via GitHub
divijvaidya commented on PR #14242: URL: https://github.com/apache/kafka/pull/14242#issuecomment-1778956865 Hey @ocadaruma This is an important change, so I despite low engagement on this from me, I do believe that this change is critical. Let's think carefully about this. I

[jira] [Created] (KAFKA-15681) Add support of client-metrics in kafka-configs.sh

2023-10-25 Thread Apoorv Mittal (Jira)
Apoorv Mittal created KAFKA-15681: - Summary: Add support of client-metrics in kafka-configs.sh Key: KAFKA-15681 URL: https://issues.apache.org/jira/browse/KAFKA-15681 Project: Kafka Issue

Re: [PR] MINOR: added compatibility for MinGW [kafka]

2023-10-25 Thread via GitHub
divijvaidya commented on code in PR #13321: URL: https://github.com/apache/kafka/pull/13321#discussion_r1371485060 ## bin/kafka-run-class.sh: ## @@ -229,7 +230,7 @@ if [ -z "$KAFKA_LOG4J_OPTS" ]; then # Log to console. This is a tool.

Re: [PR] MINOR: Rewrite the meta.properties handling code in Java and fix some issues [kafka]

2023-10-25 Thread via GitHub
divijvaidya commented on code in PR #14628: URL: https://github.com/apache/kafka/pull/14628#discussion_r1371459493 ## metadata/src/main/java/org/apache/kafka/metadata/properties/PropertiesUtils.java: ## @@ -0,0 +1,102 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

Re: [PR] KAFKA-14133: Move StreamsMetricsImplTest to Mockito [kafka]

2023-10-25 Thread via GitHub
clolov commented on PR #14623: URL: https://github.com/apache/kafka/pull/14623#issuecomment-1778922834 Local run of the failing tests passes: ``` > Configure project : Starting build with version 3.7.0-SNAPSHOT (commit id 79afb498) using Gradle 8.3, Java 1.8 and Scala 2.13.12

Re: [PR] KAFKA-15627: KIP-951's Leader discovery optimisations on the client [kafka]

2023-10-25 Thread via GitHub
msn-tldr commented on code in PR #14564: URL: https://github.com/apache/kafka/pull/14564#discussion_r1371479753 ## clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java: ## @@ -614,13 +618,28 @@ private void handleProduceResponse(ClientResponse

[jira] [Assigned] (KAFKA-15364) Handle log directory failure in the Controller

2023-10-25 Thread Omnia Ibrahim (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Omnia Ibrahim reassigned KAFKA-15364: - Assignee: Omnia Ibrahim > Handle log directory failure in the Controller >

Re: [PR] Kafka 15680 [kafka]

2023-10-25 Thread via GitHub
kumarpritam863 commented on PR #14630: URL: https://github.com/apache/kafka/pull/14630#issuecomment-1778892134 @C0urante Can you please review? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] MINOR: added compatibility for MinGW [kafka]

2023-10-25 Thread via GitHub
maniekes commented on code in PR #13321: URL: https://github.com/apache/kafka/pull/13321#discussion_r1371452345 ## bin/kafka-run-class.sh: ## @@ -229,7 +230,7 @@ if [ -z "$KAFKA_LOG4J_OPTS" ]; then # Log to console. This is a tool.

Re: [PR] KAFKA-15627: KIP-951's Leader discovery optimisations on the client [kafka]

2023-10-25 Thread via GitHub
AndrewJSchofield commented on code in PR #14564: URL: https://github.com/apache/kafka/pull/14564#discussion_r1371441595 ## clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java: ## @@ -685,6 +711,13 @@ private void completeBatch(ProducerBatch batch,

[jira] [Commented] (KAFKA-15388) Handle topics that were having compaction as retention earlier are changed to delete only retention policy and onboarded to tiered storage.

2023-10-25 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17779426#comment-17779426 ] Divij Vaidya commented on KAFKA-15388: -- Hey [~goyarpit]  I would suggest the following steps for

Re: [PR] KIP-951: protocol changes [kafka]

2023-10-25 Thread via GitHub
msn-tldr commented on PR #14627: URL: https://github.com/apache/kafka/pull/14627#issuecomment-1778870153 @chb2ab lgtm, thanks for raising this! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[jira] [Updated] (KAFKA-15671) Flaky test RemoteIndexCacheTest.testClearCacheAndIndexFilesWhenResizeCache

2023-10-25 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya updated KAFKA-15671: - Fix Version/s: 3.7.0 > Flaky test

[jira] [Resolved] (KAFKA-15671) Flaky test RemoteIndexCacheTest.testClearCacheAndIndexFilesWhenResizeCache

2023-10-25 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya resolved KAFKA-15671. -- Resolution: Fixed > Flaky test

Re: [PR] KAFKA-15671:Fix NoSuchFileException from Files.walk [kafka]

2023-10-25 Thread via GitHub
divijvaidya merged PR #14622: URL: https://github.com/apache/kafka/pull/14622 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] KAFKA-15671:Fix NoSuchFileException from Files.walk [kafka]

2023-10-25 Thread via GitHub
divijvaidya commented on PR #14622: URL: https://github.com/apache/kafka/pull/14622#issuecomment-1778850349 Unrelated test failures: ``` [Build / JDK 21 and Scala 2.13 /

Re: [PR] MINOR: added compatibility for MinGW [kafka]

2023-10-25 Thread via GitHub
divijvaidya commented on code in PR #13321: URL: https://github.com/apache/kafka/pull/13321#discussion_r1371421492 ## bin/kafka-run-class.sh: ## @@ -229,7 +230,7 @@ if [ -z "$KAFKA_LOG4J_OPTS" ]; then # Log to console. This is a tool.

Re: [PR] KAFKA-15627: KIP-951's Leader discovery optimisations on the client [kafka]

2023-10-25 Thread via GitHub
msn-tldr commented on code in PR #14564: URL: https://github.com/apache/kafka/pull/14564#discussion_r1370368421 ## clients/src/main/java/org/apache/kafka/clients/MetadataCache.java: ## @@ -150,7 +150,7 @@ MetadataCache mergeWith(String newClusterId, // We want the most

Re: [PR] KAFKA-15627: KIP-951's Leader discovery optimisations on the client [kafka]

2023-10-25 Thread via GitHub
msn-tldr commented on code in PR #14564: URL: https://github.com/apache/kafka/pull/14564#discussion_r1370368421 ## clients/src/main/java/org/apache/kafka/clients/MetadataCache.java: ## @@ -150,7 +150,7 @@ MetadataCache mergeWith(String newClusterId, // We want the most

Re: [PR] KAFKA-15627: KIP-951's Leader discovery optimisations on the client [kafka]

2023-10-25 Thread via GitHub
msn-tldr commented on code in PR #14564: URL: https://github.com/apache/kafka/pull/14564#discussion_r1370368421 ## clients/src/main/java/org/apache/kafka/clients/MetadataCache.java: ## @@ -150,7 +150,7 @@ MetadataCache mergeWith(String newClusterId, // We want the most

Re: [PR] KAFKA-15627: KIP-951's Leader discovery optimisations on the client [kafka]

2023-10-25 Thread via GitHub
msn-tldr commented on code in PR #14564: URL: https://github.com/apache/kafka/pull/14564#discussion_r1370368421 ## clients/src/main/java/org/apache/kafka/clients/MetadataCache.java: ## @@ -150,7 +150,7 @@ MetadataCache mergeWith(String newClusterId, // We want the most

Re: [PR] KAFKA-15355: Message schema changes [kafka]

2023-10-25 Thread via GitHub
soarez commented on code in PR #14290: URL: https://github.com/apache/kafka/pull/14290#discussion_r1371390386 ## metadata/src/main/resources/common/metadata/PartitionRecord.json: ## @@ -47,6 +47,13 @@ "about": "The eligible leader replicas of this partition." }, {

Re: [PR] KAFKA-13352: Kafka Client does not support passwords starting with number in jaas config [kafka]

2023-10-25 Thread via GitHub
rd-dreich commented on PR #11430: URL: https://github.com/apache/kafka/pull/11430#issuecomment-1778795647 Hi @dongjinleekr and @rajinisivaram. Are there any news regarding this PR? We're currently dealing with this issue due to some keycloak limitations and it would be great if this bug

Re: [PR] KAFKA-15355: Message schema changes [kafka]

2023-10-25 Thread via GitHub
soarez commented on code in PR #14290: URL: https://github.com/apache/kafka/pull/14290#discussion_r1371372722 ## metadata/src/main/java/org/apache/kafka/controller/PartitionChangeBuilder.java: ## @@ -54,6 +54,7 @@ public static boolean changeRecordIsNoOp(PartitionChangeRecord

Re: [PR] KAFKA-15627: KIP-951's Leader discovery optimisations on the client [kafka]

2023-10-25 Thread via GitHub
msn-tldr commented on code in PR #14564: URL: https://github.com/apache/kafka/pull/14564#discussion_r1370531714 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractFetch.java: ## @@ -171,6 +179,15 @@ protected void handleFetchResponse(final Node

Re: [PR] KAFKA-15627: KIP-951's Leader discovery optimisations on the client [kafka]

2023-10-25 Thread via GitHub
msn-tldr commented on code in PR #14564: URL: https://github.com/apache/kafka/pull/14564#discussion_r1370531714 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractFetch.java: ## @@ -171,6 +179,15 @@ protected void handleFetchResponse(final Node

Re: [PR] MINOR: Rename log dir UUIDs [kafka]

2023-10-25 Thread via GitHub
soarez commented on code in PR #14517: URL: https://github.com/apache/kafka/pull/14517#discussion_r1371360492 ## clients/src/main/java/org/apache/kafka/common/Uuid.java: ## @@ -49,21 +49,21 @@ public class Uuid implements Comparable { /** * A UUID that is used to

[jira] [Assigned] (KAFKA-15362) Resolve offline replicas in metadata cache

2023-10-25 Thread Igor Soarez (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Soarez reassigned KAFKA-15362: --- Assignee: Igor Soarez > Resolve offline replicas in metadata cache >

Re: [PR] MINOR: avoid blocking for randomness in DefaultRecordBatchTest [kafka]

2023-10-25 Thread via GitHub
soarez commented on code in PR #14625: URL: https://github.com/apache/kafka/pull/14625#discussion_r1371337334 ## clients/src/test/java/org/apache/kafka/common/record/DefaultRecordBatchTest.java: ## @@ -63,15 +61,7 @@ import static org.mockito.Mockito.when; public class

Re: [PR] KAFKA-15082: The log retention policy doesn't take effect after altering log dir [kafka]

2023-10-25 Thread via GitHub
hudeqi commented on PR #13847: URL: https://github.com/apache/kafka/pull/13847#issuecomment-1778716871 I mentioned a [PR](https://github.com/apache/kafka/pull/13421) before, which is aimed at the problem that an abnormality in copying data may occur when performing `alterlog`, causing the

Re: [PR] KAFKA-15578: System Tests for running old protocol with new coordinator [kafka]

2023-10-25 Thread via GitHub
dajac commented on code in PR #14524: URL: https://github.com/apache/kafka/pull/14524#discussion_r1371289551 ## tests/kafkatest/services/kafka/kafka.py: ## @@ -277,6 +279,22 @@ def __init__(self, context, num_nodes, zk, security_protocol=SecurityConfig.PLAI

Re: [PR] KAFKA-15578: System Tests for running old protocol with new coordinator [kafka]

2023-10-25 Thread via GitHub
dajac commented on code in PR #14524: URL: https://github.com/apache/kafka/pull/14524#discussion_r1371278858 ## tests/kafkatest/services/kafka/kafka.py: ## @@ -277,6 +279,22 @@ def __init__(self, context, num_nodes, zk, security_protocol=SecurityConfig.PLAI

[jira] [Updated] (KAFKA-15670) KRaft controller should set inter broker listener when migration

2023-10-25 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen updated KAFKA-15670: -- Description: During ZK migrating to KRaft, before entering dual-write mode, the KRaft controller

[jira] [Updated] (KAFKA-15670) KRaft controller should set inter broker listener when migration

2023-10-25 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen updated KAFKA-15670: -- Summary: KRaft controller should set inter broker listener when migration (was: KRaft controller

Re: [PR] KAFKA-15671:Fix NoSuchFileException from Files.walk [kafka]

2023-10-25 Thread via GitHub
hudeqi commented on PR #14622: URL: https://github.com/apache/kafka/pull/14622#issuecomment-1778633171 @divijvaidya This flaky test caused by Files.walk would occasionally fail when I ran it locally repeatedly. After modifying this PR, the local repeated execution has become normal in my

Re: [PR] KAFKA-15671:Fix NoSuchFileException from Files.walk [kafka]

2023-10-25 Thread via GitHub
hudeqi closed pull request #14622: KAFKA-15671:Fix NoSuchFileException from Files.walk URL: https://github.com/apache/kafka/pull/14622 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] KAFKA-15608: Assign lastet leader eopch and offset checkpoint to future log when replacing current log [kafka]

2023-10-25 Thread via GitHub
hudeqi commented on PR #14553: URL: https://github.com/apache/kafka/pull/14553#issuecomment-1778612496 > > I tested this process on the latest trunk branch and it did not reproduce. My process is: create a topic with 1 partition, then write traffic and stop writing. For example, the

Re: [PR] KAFKA-15608: Assign lastet leader eopch and offset checkpoint to future log when replacing current log [kafka]

2023-10-25 Thread via GitHub
drawxy commented on code in PR #14553: URL: https://github.com/apache/kafka/pull/14553#discussion_r1371202846 ## core/src/main/scala/kafka/log/LogManager.scala: ## @@ -1156,6 +1156,9 @@ class LogManager(logDirs: Seq[File], // we will add metrics back after sourceLog

<    1   2   3   >