Re: [PR] MINOR: Re-add Uniform assignor to the default ones [kafka]

2024-01-29 Thread via GitHub
dajac merged PR #15269: URL: https://github.com/apache/kafka/pull/15269 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

Re: [PR] KAFKA-16095: Update list group state type filter to include the states for the new consumer group type [kafka]

2024-01-29 Thread via GitHub
dajac commented on code in PR #15211: URL: https://github.com/apache/kafka/pull/15211#discussion_r1469217586 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/classic/ClassicGroup.java: ## @@ -956,6 +956,20 @@ public Optional offsetExpirationCondition() {

Re: [PR] KAFKA-14517:Implement regex subscriptions [kafka]

2024-01-29 Thread via GitHub
dajac commented on PR #14327: URL: https://github.com/apache/kafka/pull/14327#issuecomment-1914199405 Hey @JimmyWang6. I am really sorry for the delay on this pull request. We have too many other things to finish before I could really focus on it. I will have more time in the coming weeks.

Re: [PR] KAFKA-14517: Implement regex subscriptions [kafka]

2024-01-29 Thread via GitHub
dajac commented on PR #14327: URL: https://github.com/apache/kafka/pull/14327#issuecomment-1914202236 As a first step, it would be great if we could keep this pull request focused on the RPC and its implementation on the server side. I would extract the command line tool part into its own p

[jira] [Updated] (KAFKA-16017) Checkpointed offset is incorrect when task is revived and restoring

2024-01-29 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-16017: Fix Version/s: 3.5.3 3.4.2 3.6.2 > Checkpointed offs

Re: [PR] KAFKA-14517: Implement regex subscriptions [kafka]

2024-01-29 Thread via GitHub
Phuc-Hong-Tran commented on PR #14327: URL: https://github.com/apache/kafka/pull/14327#issuecomment-1914228011 @dajac I have a question regarding the usage of RE2J on the server side. Shouldn't RE2J be the primary engine that will be used for compiling regular expressions. Why is it not in

Re: [PR] KAFKA-14517: Implement regex subscriptions [kafka]

2024-01-29 Thread via GitHub
dajac commented on code in PR #14327: URL: https://github.com/apache/kafka/pull/14327#discussion_r1469244568 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -1040,6 +1051,7 @@ private CoordinatorResult consumerGr

Re: [PR] KAFKA-14517: Implement regex subscriptions [kafka]

2024-01-29 Thread via GitHub
dajac commented on PR #14327: URL: https://github.com/apache/kafka/pull/14327#issuecomment-1914230236 > @dajac I have a question regarding the usage of RE2J on the server side. Shouldn't RE2J be the primary engine that will be used for compiling regular expressions. Why is it not in the sco

Re: [PR] KAFKA-16095: Update list group state type filter to include the states for the new consumer group type [kafka]

2024-01-29 Thread via GitHub
DL1231 commented on code in PR #15211: URL: https://github.com/apache/kafka/pull/15211#discussion_r1469259935 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/classic/ClassicGroup.java: ## @@ -956,6 +956,20 @@ public Optional offsetExpirationCondition() {

Re: [PR] KAFKA-16066: remove apacheds to 2.0.0.AM27(change apache kerby) [kafka]

2024-01-29 Thread via GitHub
mimaison commented on PR #15277: URL: https://github.com/apache/kafka/pull/15277#issuecomment-1914302395 It looks like something is still not quite right, the latest [CI run](https://ci-builds.apache.org/job/Kafka/job/kafka-pr/job/PR-15277/4/testReport/) has many failures with: ``` [2

Re: [PR] KAFKA-14517: Implement regex subscriptions [kafka]

2024-01-29 Thread via GitHub
Phuc-Hong-Tran commented on code in PR #14327: URL: https://github.com/apache/kafka/pull/14327#discussion_r1469332580 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -1573,6 +1589,33 @@ private void updateGroupsByTopics(

Re: [PR] KAFKA-16095: Update list group state type filter to include the states for the new consumer group type [kafka]

2024-01-29 Thread via GitHub
dajac commented on code in PR #15211: URL: https://github.com/apache/kafka/pull/15211#discussion_r1469341470 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/classic/ClassicGroup.java: ## @@ -956,6 +956,20 @@ public Optional offsetExpirationCondition() {

[PR] MINOR: Clean up core metrics, migration, network, raft, security, serializer, tools, utils, and zookeeper modules [kafka]

2024-01-29 Thread via GitHub
jlprat opened a new pull request, #15279: URL: https://github.com/apache/kafka/pull/15279 Second follow up from https://github.com/apache/kafka/pull/15252 This PR cleans up: metrics, migration, network, raft, security, serializer, tools, utils, and zookeeper package classes - M

[jira] [Created] (KAFKA-16201) Kafka exception - org.apache.kafka.common.errors.NotLeaderOrFollowerException

2024-01-29 Thread Yogesh (Jira)
Yogesh created KAFKA-16201: -- Summary: Kafka exception - org.apache.kafka.common.errors.NotLeaderOrFollowerException Key: KAFKA-16201 URL: https://issues.apache.org/jira/browse/KAFKA-16201 Project: Kafka

Re: [PR] MINOR: Uniformize error handling/transformation in GroupCoordinatorService [kafka]

2024-01-29 Thread via GitHub
dajac commented on code in PR #15196: URL: https://github.com/apache/kafka/pull/15196#discussion_r1469367828 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinatorService.java: ## @@ -1099,29 +1055,48 @@ private static boolean isGroupIdNotEmpty(St

Re: [PR] Initial support for OpenJDK CRaC snapshotting [kafka]

2024-01-29 Thread via GitHub
rvansa commented on PR #13619: URL: https://github.com/apache/kafka/pull/13619#issuecomment-1914437381 Hi @tzvetkovg , there was not so much reaction so far from the community, so basically - we need your feedback. You can use the artifacts at https://mvnrepository.com/artifact/io.github.cr

[jira] [Commented] (KAFKA-4759) Add support for subnet masks in SimpleACLAuthorizer

2024-01-29 Thread Szymon Scharmach (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17811821#comment-17811821 ] Szymon Scharmach commented on KAFKA-4759: - Hi, [~jeqo]  - KIP is here: [https:/

Re: [PR] KAFKA-16095: Update list group state type filter to include the states for the new consumer group type [kafka]

2024-01-29 Thread via GitHub
DL1231 commented on code in PR #15211: URL: https://github.com/apache/kafka/pull/15211#discussion_r1469457414 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroup.java: ## @@ -67,14 +68,22 @@ public enum ConsumerGroupState { pri

[PR] MINOR: add client info in authentication error log [kafka]

2024-01-29 Thread via GitHub
showuon opened a new pull request, #15280: URL: https://github.com/apache/kafka/pull/15280 When client authenticate failed, the server will log with: the client IP address only. The the IP address sometimes cannot represent a specific user, especially if there is proxy between client and se

Re: [PR] MINOR: add client info in authentication error log [kafka]

2024-01-29 Thread via GitHub
showuon commented on code in PR #15280: URL: https://github.com/apache/kafka/pull/15280#discussion_r1469530712 ## clients/src/main/java/org/apache/kafka/common/security/oauthbearer/internals/OAuthBearerSaslServer.java: ## @@ -159,10 +159,15 @@ private byte[] process(String token

Re: [PR] MINOR: add client info in authentication error log [kafka]

2024-01-29 Thread via GitHub
showuon commented on code in PR #15280: URL: https://github.com/apache/kafka/pull/15280#discussion_r1469531952 ## clients/src/main/java/org/apache/kafka/common/security/plain/internals/PlainSaslServer.java: ## @@ -93,20 +93,20 @@ public byte[] evaluateResponse(byte[] responseByt

Re: [PR] MINOR: add client info in authentication error log [kafka]

2024-01-29 Thread via GitHub
showuon commented on code in PR #15280: URL: https://github.com/apache/kafka/pull/15280#discussion_r1469531952 ## clients/src/main/java/org/apache/kafka/common/security/plain/internals/PlainSaslServer.java: ## @@ -93,20 +93,20 @@ public byte[] evaluateResponse(byte[] responseByt

Re: [PR] MINOR: add client info in authentication error log [kafka]

2024-01-29 Thread via GitHub
showuon commented on code in PR #15280: URL: https://github.com/apache/kafka/pull/15280#discussion_r1469537246 ## clients/src/main/java/org/apache/kafka/common/security/scram/internals/ScramSaslServer.java: ## @@ -127,10 +127,10 @@ public byte[] evaluateResponse(byte[] response)

Re: [PR] MINOR: add client info in authentication error log [kafka]

2024-01-29 Thread via GitHub
showuon commented on code in PR #15280: URL: https://github.com/apache/kafka/pull/15280#discussion_r1469542360 ## clients/src/main/java/org/apache/kafka/common/security/scram/internals/ScramSaslServer.java: ## @@ -143,7 +143,7 @@ public byte[] evaluateResponse(byte[] response) t

Re: [PR] KAFKA-16198: Reconciliation may lose partitions when topic metadata is delayed [kafka]

2024-01-29 Thread via GitHub
lucasbru commented on code in PR #15271: URL: https://github.com/apache/kafka/pull/15271#discussion_r1469529850 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -208,22 +209,18 @@ public class MembershipManagerImpl implements

Re: [PR] KAFKA-14505; [6/N] Avoid recheduling callback in request thread [kafka]

2024-01-29 Thread via GitHub
dajac commented on PR #15176: URL: https://github.com/apache/kafka/pull/15176#issuecomment-1914678256 @jolshan I reworked the PR based on https://github.com/apache/kafka/pull/15087. It is quite different from the previous one. Please take a look when you get a chance. -- This is an autom

Re: [PR] KAFKA-14517: Implement regex subscriptions [kafka]

2024-01-29 Thread via GitHub
dajac commented on code in PR #14327: URL: https://github.com/apache/kafka/pull/14327#discussion_r1469602233 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -1573,6 +1589,33 @@ private void updateGroupsByTopics(

Re: [PR] KAFKA-15942: Implement ConsumerInterceptors in the async consumer [kafka]

2024-01-29 Thread via GitHub
cadonna commented on code in PR #15000: URL: https://github.com/apache/kafka/pull/15000#discussion_r1469619037 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -271,6 +283,7 @@ public CompletableFuture maybeAutoCommitAllConsu

Re: [PR] KAFKA-15675: Improve worker liveness check during Connect integration tests [kafka]

2024-01-29 Thread via GitHub
C0urante commented on PR #15249: URL: https://github.com/apache/kafka/pull/15249#issuecomment-1914760363 [Last CI run](https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fkafka-pr/detail/PR-15249/6/tests/) completed with no failures in the `ConnectorRestartApiIntegrationTest` su

Re: [PR] KAFKA-10816: Initialize REST endpoints only after the herder has started [kafka]

2024-01-29 Thread via GitHub
C0urante commented on PR #9826: URL: https://github.com/apache/kafka/pull/9826#issuecomment-1914768311 Better late than never! https://cwiki.apache.org/confluence/display/KAFKA/KIP-1017%3A+Health+check+endpoint+for+Kafka+Connect -- This is an automated message from the Apache Git Service.

Re: [PR] KAFKA-10816: Use root resource as readiness and health probe for Connect distributed mode [kafka]

2024-01-29 Thread via GitHub
C0urante commented on PR #10520: URL: https://github.com/apache/kafka/pull/10520#issuecomment-1914768704 Better late than never! https://cwiki.apache.org/confluence/display/KAFKA/KIP-1017%3A+Health+check+endpoint+for+Kafka+Connect -- This is an automated message from the Apache Git Servic

Re: [PR] KAFKA-15942: Implement ConsumerInterceptors in the async consumer [kafka]

2024-01-29 Thread via GitHub
cadonna commented on code in PR #15000: URL: https://github.com/apache/kafka/pull/15000#discussion_r1469657659 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumerTest.java: ## @@ -848,6 +853,70 @@ public void testWakeupCommitted() { a

Re: [PR] KAFKA-16095: Update list group state type filter to include the states for the new consumer group type [kafka]

2024-01-29 Thread via GitHub
DL1231 commented on code in PR #15211: URL: https://github.com/apache/kafka/pull/15211#discussion_r1469662118 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/classic/ClassicGroup.java: ## @@ -956,6 +956,20 @@ public Optional offsetExpirationCondition() {

Re: [PR] KAFKA-16095: Update list group state type filter to include the states for the new consumer group type [kafka]

2024-01-29 Thread via GitHub
DL1231 commented on code in PR #15211: URL: https://github.com/apache/kafka/pull/15211#discussion_r1469662425 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroup.java: ## @@ -739,6 +748,19 @@ public Optional offsetExpirationCondition() {

Re: [PR] KAFKA-16082: Avoid resuming future replica if current replica is in the same directory [kafka]

2024-01-29 Thread via GitHub
OmniaGM commented on PR #15136: URL: https://github.com/apache/kafka/pull/15136#issuecomment-1914798524 Hi, @rondagostino, @showuon and @cmccabe can one of you have a look into this pr please and merge it if you are happy with the changes? -- This is an automated message from the Apache

Re: [PR] KAFKA-16162: resend broker registration on metadata update to IBP 3.7-IV2 [kafka]

2024-01-29 Thread via GitHub
gaurav-narula commented on PR #15270: URL: https://github.com/apache/kafka/pull/15270#issuecomment-1914817144 > Looks straightforward fix! however the pipeline is failing with > > ``` > > Task :core:compileTestScala > [2024-01-26T15:19:03.811Z] [Error] /home/jenkins/workspace/K

Re: [PR] KAFKA-16157: fix topic recreation handling with offline disks [kafka]

2024-01-29 Thread via GitHub
OmniaGM commented on code in PR #15263: URL: https://github.com/apache/kafka/pull/15263#discussion_r1469680659 ## core/src/main/scala/kafka/cluster/Partition.scala: ## @@ -874,7 +874,13 @@ class Partition(val topicPartition: TopicPartition, private def createLogInAssignedDire

Re: [PR] KAFKA-16157: fix topic recreation handling with offline disks [kafka]

2024-01-29 Thread via GitHub
OmniaGM commented on code in PR #15263: URL: https://github.com/apache/kafka/pull/15263#discussion_r1469680659 ## core/src/main/scala/kafka/cluster/Partition.scala: ## @@ -874,7 +874,13 @@ class Partition(val topicPartition: TopicPartition, private def createLogInAssignedDire

Re: [PR] KAFKA-16195: ignore metadata.log.dir failure in ZK mode [kafka]

2024-01-29 Thread via GitHub
OmniaGM commented on PR #15262: URL: https://github.com/apache/kafka/pull/15262#issuecomment-1914824513 Tests Compilation is failing with ``` [2024-01-26T16:18:05.648Z] > Task :core:compileTestScala [2024-01-26T16:18:05.648Z] [Error] /home/jenkins/jenkins-agent/712657a4/workspace/K

[jira] [Created] (KAFKA-16202) Extra dot in error message in producer

2024-01-29 Thread Mickael Maison (Jira)
Mickael Maison created KAFKA-16202: -- Summary: Extra dot in error message in producer Key: KAFKA-16202 URL: https://issues.apache.org/jira/browse/KAFKA-16202 Project: Kafka Issue Type: Improv

Re: [PR] KAFKA-16157: fix topic recreation handling with offline disks [kafka]

2024-01-29 Thread via GitHub
OmniaGM commented on code in PR #15263: URL: https://github.com/apache/kafka/pull/15263#discussion_r1469691076 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -183,7 +183,7 @@ object HostedPartition { /** * This broker hosts the partition, but it is in an

Re: [PR] KAFKA-16157: fix topic recreation handling with offline disks [kafka]

2024-01-29 Thread via GitHub
OmniaGM commented on code in PR #15263: URL: https://github.com/apache/kafka/pull/15263#discussion_r1469696139 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -183,7 +183,7 @@ object HostedPartition { /** * This broker hosts the partition, but it is in an

Re: [PR] KAFKA-15575: Begin enforcing 'tasks.max' property for connectors [kafka]

2024-01-29 Thread via GitHub
C0urante commented on code in PR #15180: URL: https://github.com/apache/kafka/pull/15180#discussion_r1469698231 ## connect/runtime/src/main/java/org/apache/kafka/connect/runtime/Worker.java: ## @@ -620,6 +650,11 @@ private boolean startTask( try (LoaderSwap loader

Re: [PR] KAFKA-15575: Begin enforcing 'tasks.max' property for connectors [kafka]

2024-01-29 Thread via GitHub
C0urante commented on PR #15180: URL: https://github.com/apache/kafka/pull/15180#issuecomment-1914843700 @gharris1727 I think I've addressed all outstanding comments; is there anything else you'd like to discuss for this one? -- This is an automated message from the Apache Git Service. To

[PR] Fix to include runtime dependencies in clients [kafka]

2024-01-29 Thread via GitHub
apoorvmittal10 opened a new pull request, #15281: URL: https://github.com/apache/kafka/pull/15281 Fix in the following https://github.com/apache/kafka/pull/15127 introduced a regression for `kafka-clients` pom where runtime dependencies were not included in the `kafka-clients` pom. The PR h

[jira] [Created] (KAFKA-16203) AutoCommit of empty offsets blocks following requests due to inflight flag

2024-01-29 Thread Lianet Magrans (Jira)
Lianet Magrans created KAFKA-16203: -- Summary: AutoCommit of empty offsets blocks following requests due to inflight flag Key: KAFKA-16203 URL: https://issues.apache.org/jira/browse/KAFKA-16203 Projec

Re: [PR] KAFKA-15942: Implement ConsumerInterceptors in the async consumer [kafka]

2024-01-29 Thread via GitHub
lucasbru commented on code in PR #15000: URL: https://github.com/apache/kafka/pull/15000#discussion_r1469714382 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumerTest.java: ## @@ -848,6 +853,70 @@ public void testWakeupCommitted() {

[jira] [Updated] (KAFKA-16203) AutoCommit of empty offsets blocks following requests due to inflight flag

2024-01-29 Thread Lianet Magrans (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianet Magrans updated KAFKA-16203: --- Description: The logic for auto-committing offsets completes without generating a request, b

Re: [PR] KAFKA-16203: Fix bug on auto-commit empty offsets blocking inflight [kafka]

2024-01-29 Thread via GitHub
lianetm commented on PR #15282: URL: https://github.com/apache/kafka/pull/15282#issuecomment-1914888375 @dajac could you please add the kip-848 label when you have a chance to take a look at this? Thanks for reporting it! -- This is an automated message from the Apache Git Service. To res

Re: [PR] KAFKA-16095: Update list group state type filter to include the states for the new consumer group type [kafka]

2024-01-29 Thread via GitHub
dajac merged PR #15211: URL: https://github.com/apache/kafka/pull/15211 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.or

[jira] [Resolved] (KAFKA-16095) Update list group state type filter to include the states for the new consumer group type

2024-01-29 Thread David Jacot (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-16095. - Fix Version/s: 3.8.0 Resolution: Fixed > Update list group state type filter to include t

Re: [PR] KAFKA-15307: update/note deprecated configs [kafka]

2024-01-29 Thread via GitHub
Cerchie commented on code in PR #14360: URL: https://github.com/apache/kafka/pull/14360#discussion_r1469755433 ## docs/streams/developer-guide/config-streams.html: ## @@ -257,7 +258,12 @@ num.standby.replicasThe maximum number of records to buffer per partition. 10

Re: [PR] KAFKA-15307: update/note deprecated configs [kafka]

2024-01-29 Thread via GitHub
Cerchie commented on code in PR #14360: URL: https://github.com/apache/kafka/pull/14360#discussion_r1469760873 ## docs/streams/developer-guide/config-streams.html: ## @@ -1010,6 +1016,18 @@ topology.optimization + windowed.inner.class.serde + + + +

Re: [PR] KAFKA-16189; Extend admin to support ConsumerGroupDescribe API [kafka]

2024-01-29 Thread via GitHub
nizhikov commented on PR #15253: URL: https://github.com/apache/kafka/pull/15253#issuecomment-1914941776 Hello @dajac Right now I have #15256 which moves `DescribeConsumerGroupTest` and `ConsumerGroupCommandTest` to tools module. You are modifying those tests in PR. My PR is re

Re: [PR] KAFKA-16186: Broker metrics for client telemetry (KIP-714) [kafka]

2024-01-29 Thread via GitHub
apoorvmittal10 commented on code in PR #15251: URL: https://github.com/apache/kafka/pull/15251#discussion_r1469784272 ## server/src/main/java/org/apache/kafka/server/ClientMetricsManager.java: ## @@ -493,4 +519,123 @@ public void run() { } } } + +/

[jira] [Created] (KAFKA-16204) Stray file core/00000000000000000001.snapshot created when running core tests

2024-01-29 Thread Mickael Maison (Jira)
Mickael Maison created KAFKA-16204: -- Summary: Stray file core/0001.snapshot created when running core tests Key: KAFKA-16204 URL: https://issues.apache.org/jira/browse/KAFKA-16204 Pro

Re: [PR] KAFKA-16186: Broker metrics for client telemetry (KIP-714) [kafka]

2024-01-29 Thread via GitHub
apoorvmittal10 commented on code in PR #15251: URL: https://github.com/apache/kafka/pull/15251#discussion_r1469798809 ## server/src/main/java/org/apache/kafka/server/ClientMetricsManager.java: ## @@ -493,4 +519,123 @@ public void run() { } } } + +/

Re: [PR] MINOR: Clean up core server classes [kafka]

2024-01-29 Thread via GitHub
mimaison commented on code in PR #15272: URL: https://github.com/apache/kafka/pull/15272#discussion_r1469799402 ## core/src/main/scala/kafka/server/AutoTopicCreationManager.scala: ## @@ -50,6 +50,10 @@ trait AutoTopicCreationManager { object AutoTopicCreationManager { + /*

Re: [PR] MINOR: Clean up core server classes [kafka]

2024-01-29 Thread via GitHub
jlprat commented on code in PR #15272: URL: https://github.com/apache/kafka/pull/15272#discussion_r1469806761 ## core/src/main/scala/kafka/server/AutoTopicCreationManager.scala: ## @@ -50,6 +50,10 @@ trait AutoTopicCreationManager { object AutoTopicCreationManager { + /**

Re: [PR] KAFKA-16189; Extend admin to support ConsumerGroupDescribe API [kafka]

2024-01-29 Thread via GitHub
dajac commented on PR #15253: URL: https://github.com/apache/kafka/pull/15253#issuecomment-1915007420 Thanks for letting me know, @nizhikov. I can definitely adapt my PR when https://github.com/apache/kafka/pull/15256 gets merged. However, I cannot wait too long because this PR is needed fo

Re: [PR] MINOR: Clean up core metrics, migration, network, raft, security, serializer, tools, utils, and zookeeper modules [kafka]

2024-01-29 Thread via GitHub
mimaison commented on code in PR #15279: URL: https://github.com/apache/kafka/pull/15279#discussion_r1469814528 ## core/src/main/scala/kafka/tools/ConsoleConsumer.scala: ## @@ -37,6 +37,7 @@ import org.apache.kafka.common.utils.Time import org.apache.kafka.common.utils.Utils i

Re: [PR] KAFKA-14616: Fix stray replica of recreated topics in KRaft mode [kafka]

2024-01-29 Thread via GitHub
OmniaGM commented on PR #15230: URL: https://github.com/apache/kafka/pull/15230#issuecomment-1915041492 > Question: why can't we delete these disappeared(deleted) logs directly? I think this can only happen when topic deleting while the node is offline. If so, then deleting them should be f

Re: [PR] KAFKA-16186: Broker metrics for client telemetry (KIP-714) [kafka]

2024-01-29 Thread via GitHub
apoorvmittal10 commented on code in PR #15251: URL: https://github.com/apache/kafka/pull/15251#discussion_r1469835995 ## server/src/main/java/org/apache/kafka/server/ClientMetricsManager.java: ## @@ -493,4 +519,123 @@ public void run() { } } } + +/

Re: [PR] KAFKA-16186: Broker metrics for client telemetry (KIP-714) [kafka]

2024-01-29 Thread via GitHub
apoorvmittal10 commented on code in PR #15251: URL: https://github.com/apache/kafka/pull/15251#discussion_r1469836415 ## server/src/main/java/org/apache/kafka/server/ClientMetricsManager.java: ## @@ -493,4 +519,123 @@ public void run() { } } } + +/

Re: [PR] KAFKA-16186: Broker metrics for client telemetry (KIP-714) [kafka]

2024-01-29 Thread via GitHub
apoorvmittal10 commented on code in PR #15251: URL: https://github.com/apache/kafka/pull/15251#discussion_r1469843003 ## server/src/main/java/org/apache/kafka/server/ClientMetricsManager.java: ## @@ -493,4 +519,123 @@ public void run() { } } } + +/

Re: [PR] KAFKA-16186: Broker metrics for client telemetry (KIP-714) [kafka]

2024-01-29 Thread via GitHub
apoorvmittal10 commented on code in PR #15251: URL: https://github.com/apache/kafka/pull/15251#discussion_r1469845294 ## server/src/main/java/org/apache/kafka/server/ClientMetricsManager.java: ## @@ -288,6 +307,9 @@ private ClientMetricsInstance createClientInstanceAndUpdateCac

[jira] [Assigned] (KAFKA-16194) KafkaConsumer.groupMetadata() should be correct when first records are returned

2024-01-29 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True reassigned KAFKA-16194: - Assignee: Bruno Cadonna > KafkaConsumer.groupMetadata() should be correct when first records ar

Re: [PR] KAFKA-16186: Broker metrics for client telemetry (KIP-714) [kafka]

2024-01-29 Thread via GitHub
apoorvmittal10 commented on code in PR #15251: URL: https://github.com/apache/kafka/pull/15251#discussion_r1469849574 ## server/src/main/java/org/apache/kafka/server/ClientMetricsManager.java: ## @@ -493,4 +519,123 @@ public void run() { } } } + +/

[jira] [Updated] (KAFKA-16194) KafkaConsumer.groupMetadata() should be correct when first records are returned

2024-01-29 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-16194: -- Component/s: consumer > KafkaConsumer.groupMetadata() should be correct when first records are > retu

[jira] [Updated] (KAFKA-16194) KafkaConsumer.groupMetadata() should be correct when first records are returned

2024-01-29 Thread Kirk True (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirk True updated KAFKA-16194: -- Labels: consumer-threading-refactor (was: ) > KafkaConsumer.groupMetadata() should be correct when fi

Re: [PR] KAFKA-16186: Broker metrics for client telemetry (KIP-714) [kafka]

2024-01-29 Thread via GitHub
apoorvmittal10 commented on code in PR #15251: URL: https://github.com/apache/kafka/pull/15251#discussion_r1469851552 ## server/src/test/java/org/apache/kafka/server/ClientMetricsManagerTest.java: ## @@ -1023,5 +1177,18 @@ public void testCacheExpirationTaskCancelledOnInstanceU

Re: [PR] KAFKA-16186: Broker metrics for client telemetry (KIP-714) [kafka]

2024-01-29 Thread via GitHub
apoorvmittal10 commented on code in PR #15251: URL: https://github.com/apache/kafka/pull/15251#discussion_r1469854621 ## server/src/test/java/org/apache/kafka/server/ClientMetricsManagerTest.java: ## @@ -299,8 +332,8 @@ public void testGetTelemetrySameClientImmediateRetryAfterP

Re: [PR] KAFKA-15853: Move PasswordEncoder to server-common [kafka]

2024-01-29 Thread via GitHub
mimaison commented on PR #15246: URL: https://github.com/apache/kafka/pull/15246#issuecomment-1915069486 @OmniaGM Since you've been leading in the breakdown of KafkaConfig, do you want to take a look at this PR? -- This is an automated message from the Apache Git Service. To respond to th

[jira] [Commented] (KAFKA-16204) Stray file core/00000000000000000001.snapshot created when running core tests

2024-01-29 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17811983#comment-17811983 ] Justine Olshan commented on KAFKA-16204: I've seen it when running just build +

Re: [PR] MINOR: Clean up core metrics, migration, network, raft, security, serializer, tools, utils, and zookeeper modules [kafka]

2024-01-29 Thread via GitHub
jlprat commented on code in PR #15279: URL: https://github.com/apache/kafka/pull/15279#discussion_r1469908118 ## core/src/main/scala/kafka/tools/ConsoleConsumer.scala: ## @@ -37,6 +37,7 @@ import org.apache.kafka.common.utils.Time import org.apache.kafka.common.utils.Utils imp

Re: [PR] MINOR: Clean up core metrics, migration, network, raft, security, serializer, tools, utils, and zookeeper modules [kafka]

2024-01-29 Thread via GitHub
mimaison commented on code in PR #15279: URL: https://github.com/apache/kafka/pull/15279#discussion_r1469921954 ## core/src/main/scala/kafka/tools/ConsoleConsumer.scala: ## @@ -37,6 +37,7 @@ import org.apache.kafka.common.utils.Time import org.apache.kafka.common.utils.Utils i

[jira] [Created] (KAFKA-16205) Reduce number of metadata requests during hybrid mode

2024-01-29 Thread David Arthur (Jira)
David Arthur created KAFKA-16205: Summary: Reduce number of metadata requests during hybrid mode Key: KAFKA-16205 URL: https://issues.apache.org/jira/browse/KAFKA-16205 Project: Kafka Issue T

[PR] KAFKA-16205 Allow MetadataLoader to coalesce small batches [kafka]

2024-01-29 Thread via GitHub
mumrah opened a new pull request, #15283: URL: https://github.com/apache/kafka/pull/15283 Rather than immediately publishing a MetadataDelta for each commit handled from the Raft layer, this patch uses a short deferral window for accumulating changes from multiple Raft commits. A deferred e

Re: [PR] KAFKA-15853: Move PasswordEncoder to server-common [kafka]

2024-01-29 Thread via GitHub
OmniaGM commented on code in PR #15246: URL: https://github.com/apache/kafka/pull/15246#discussion_r1469969299 ## server-common/src/main/java/org/apache/kafka/security/EncryptingPasswordEncoder.java: ## @@ -0,0 +1,155 @@ +/* + * Licensed to the Apache Software Foundation (ASF) u

Re: [PR] KAFKA-16157: fix topic recreation handling with offline disks [kafka]

2024-01-29 Thread via GitHub
gnarula commented on code in PR #15263: URL: https://github.com/apache/kafka/pull/15263#discussion_r1469986094 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -183,7 +183,7 @@ object HostedPartition { /** * This broker hosts the partition, but it is in an

[jira] [Created] (KAFKA-16206) ZkConfigMigrationClient tries to delete topic configs twice

2024-01-29 Thread David Arthur (Jira)
David Arthur created KAFKA-16206: Summary: ZkConfigMigrationClient tries to delete topic configs twice Key: KAFKA-16206 URL: https://issues.apache.org/jira/browse/KAFKA-16206 Project: Kafka

[jira] [Updated] (KAFKA-16171) Controller failover during ZK migration can prevent metadata updates to ZK brokers

2024-01-29 Thread David Arthur (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur updated KAFKA-16171: - Component/s: migration (was: controller) > Controller failover during ZK mi

[jira] [Updated] (KAFKA-16205) Reduce number of metadata requests during hybrid mode

2024-01-29 Thread David Arthur (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur updated KAFKA-16205: - Component/s: migration > Reduce number of metadata requests during hybrid mode > ---

[jira] [Updated] (KAFKA-16205) Reduce number of metadata requests during hybrid mode

2024-01-29 Thread David Arthur (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur updated KAFKA-16205: - Component/s: (was: controller) > Reduce number of metadata requests during hybrid mode > ---

[jira] [Updated] (KAFKA-16205) Reduce number of metadata requests during hybrid mode

2024-01-29 Thread David Arthur (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur updated KAFKA-16205: - Component/s: controller > Reduce number of metadata requests during hybrid mode > --

[jira] [Updated] (KAFKA-16171) Controller failover during ZK migration can prevent metadata updates to ZK brokers

2024-01-29 Thread David Arthur (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur updated KAFKA-16171: - Component/s: controller > Controller failover during ZK migration can prevent metadata updates t

[jira] [Updated] (KAFKA-16206) KRaftMigrationZkWriter tries to delete deleted topic configs twice

2024-01-29 Thread David Arthur (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Arthur updated KAFKA-16206: - Summary: KRaftMigrationZkWriter tries to delete deleted topic configs twice (was: ZkConfigMigra

Re: [PR] KAFKA-16157: fix topic recreation handling with offline disks [kafka]

2024-01-29 Thread via GitHub
OmniaGM commented on code in PR #15263: URL: https://github.com/apache/kafka/pull/15263#discussion_r1469993396 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -183,7 +183,7 @@ object HostedPartition { /** * This broker hosts the partition, but it is in an

Re: [PR] KAFKA-16186: Broker metrics for client telemetry (KIP-714) [kafka]

2024-01-29 Thread via GitHub
AndrewJSchofield commented on code in PR #15251: URL: https://github.com/apache/kafka/pull/15251#discussion_r146343 ## server/src/main/java/org/apache/kafka/server/ClientMetricsManager.java: ## @@ -493,4 +519,123 @@ public void run() { } } } + +

Re: [PR] KAFKA-16157: fix topic recreation handling with offline disks [kafka]

2024-01-29 Thread via GitHub
OmniaGM commented on code in PR #15263: URL: https://github.com/apache/kafka/pull/15263#discussion_r1470001208 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -183,7 +183,7 @@ object HostedPartition { /** * This broker hosts the partition, but it is in an

Re: [PR] KAFKA-16115: Adding missing heartbeat metrics [kafka]

2024-01-29 Thread via GitHub
philipnee commented on code in PR #15216: URL: https://github.com/apache/kafka/pull/15216#discussion_r1470058217 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/metrics/KafkaConsumerMetrics.java: ## @@ -26,20 +26,20 @@ import java.util.concurrent.TimeUnit

Re: [PR] KAFKA-16115: Adding missing heartbeat metrics [kafka]

2024-01-29 Thread via GitHub
philipnee commented on code in PR #15216: URL: https://github.com/apache/kafka/pull/15216#discussion_r1470063334 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManagerTest.java: ## @@ -671,18 +718,8 @@ private ConsumerConfig config() {

Re: [PR] KAFKA-16186: Broker metrics for client telemetry (KIP-714) [kafka]

2024-01-29 Thread via GitHub
junrao commented on code in PR #15251: URL: https://github.com/apache/kafka/pull/15251#discussion_r1470043472 ## server/src/main/java/org/apache/kafka/server/ClientMetricsManager.java: ## @@ -288,6 +307,9 @@ private ClientMetricsInstance createClientInstanceAndUpdateCache(Uuid

Re: [PR] MINOR: Uniformize error handling/transformation in GroupCoordinatorService [kafka]

2024-01-29 Thread via GitHub
jolshan commented on code in PR #15196: URL: https://github.com/apache/kafka/pull/15196#discussion_r1470066073 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinatorService.java: ## @@ -523,7 +524,7 @@ public CompletableFuture listGroups(

Re: [PR] MINOR: Uniformize error handling/transformation in GroupCoordinatorService [kafka]

2024-01-29 Thread via GitHub
jolshan commented on code in PR #15196: URL: https://github.com/apache/kafka/pull/15196#discussion_r1470066073 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinatorService.java: ## @@ -523,7 +524,7 @@ public CompletableFuture listGroups(

Re: [PR] KAFKA-16115: Adding missing heartbeat metrics [kafka]

2024-01-29 Thread via GitHub
philipnee commented on PR #15216: URL: https://github.com/apache/kafka/pull/15216#issuecomment-1915374713 Hi @lucasbru - This PR is ready for review. I did a bit of refactor in this PR so not everything here is in the scope. Let me know what do you think! -- This is an automated message

Re: [PR] MINOR: Uniformize error handling/transformation in GroupCoordinatorService [kafka]

2024-01-29 Thread via GitHub
dajac commented on code in PR #15196: URL: https://github.com/apache/kafka/pull/15196#discussion_r1470072535 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinatorService.java: ## @@ -523,7 +524,7 @@ public CompletableFuture listGroups(

Re: [PR] MINOR: Uniformize error handling/transformation in GroupCoordinatorService [kafka]

2024-01-29 Thread via GitHub
jolshan commented on code in PR #15196: URL: https://github.com/apache/kafka/pull/15196#discussion_r1470078455 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinatorService.java: ## @@ -1099,29 +1055,48 @@ private static boolean isGroupIdNotEmpty(

Re: [PR] KAFKA-16198: Reconciliation may lose partitions when topic metadata is delayed [kafka]

2024-01-29 Thread via GitHub
lianetm commented on code in PR #15271: URL: https://github.com/apache/kafka/pull/15271#discussion_r1470081608 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/MembershipManagerImpl.java: ## @@ -997,11 +985,13 @@ void markReconciliationCompleted() { *

Re: [PR] KAFKA-14505; [6/N] Avoid recheduling callback in request thread [kafka]

2024-01-29 Thread via GitHub
jolshan commented on code in PR #15176: URL: https://github.com/apache/kafka/pull/15176#discussion_r1470096146 ## core/src/main/scala/kafka/coordinator/group/GroupCoordinator.scala: ## @@ -935,8 +935,12 @@ private[group] class GroupCoordinator( producerId,

  1   2   >