[jira] [Commented] (KAFKA-15169) Add tests for RemoteIndexCache

2023-10-03 Thread Arpit Goyal (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17771713#comment-17771713 ] Arpit Goyal commented on KAFKA-15169: - [~divijvaidya] [~satish.duggana] [~showuon] A

[PR] KAFKA-15169: Added TestCase in RemoteIndexCache [kafka]

2023-10-03 Thread via GitHub
iit2009060 opened a new pull request, #14482: URL: https://github.com/apache/kafka/pull/14482 **Test Cases Covered** 1. _Index Files already exist on disk but not in Cache_ i.e. RemoteIndexCache should not call remoteStorageManager to fetch it instead cache it from the local index

[PR] KAFKA-15484/General Rack Aware Assignor [kafka]

2023-10-03 Thread via GitHub
rreddy-22 opened a new pull request, #14481: URL: https://github.com/apache/kafka/pull/14481 This is part two of the Uniform Assignor, used when the subscriptions of each member in a consumer group are different. Files added are: GeneralUniformAssignmentBuilder: Contains all the assign

Re: [PR] KAFKA-14684 Replace EasyMock and PowerMock with Mockito in WorkerSinkTaskThreadedTest [kafka]

2023-10-03 Thread via GitHub
github-actions[bot] commented on PR #13951: URL: https://github.com/apache/kafka/pull/13951#issuecomment-1746079088 This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please ask a committer for review. If the PR has merge

[jira] [Assigned] (KAFKA-15535) Add documentation of "remote.log.index.file.cache.total.size.bytes" configuration property.

2023-10-03 Thread hudeqi (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hudeqi reassigned KAFKA-15535: -- Assignee: hudeqi > Add documentation of "remote.log.index.file.cache.total.size.bytes" > configurati

Re: [PR] KAFKA-14912:Add a dynamic config for remote index cache size [kafka]

2023-10-03 Thread via GitHub
showuon commented on code in PR #14381: URL: https://github.com/apache/kafka/pull/14381#discussion_r1345081564 ## storage/src/main/java/org/apache/kafka/storage/internals/log/RemoteIndexCache.java: ## @@ -112,27 +115,55 @@ public class RemoteIndexCache implements Closeable {

[jira] [Created] (KAFKA-15536) dynamically resize remoteIndexCache

2023-10-03 Thread Luke Chen (Jira)
Luke Chen created KAFKA-15536: - Summary: dynamically resize remoteIndexCache Key: KAFKA-15536 URL: https://issues.apache.org/jira/browse/KAFKA-15536 Project: Kafka Issue Type: Improvement

Re: [PR] KAFKA-15526: Simplify the LogAppendInfo class [kafka]

2023-10-03 Thread via GitHub
jolshan merged PR #14470: URL: https://github.com/apache/kafka/pull/14470 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-03 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1344900260 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumer.java: ## @@ -489,20 +623,44 @@ private Map beginningOrEndOffset(Collection

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-03 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1344899034 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumer.java: ## @@ -395,12 +523,12 @@ private void maybeThrowInvalidGroupIdExcepti

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-03 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1344897098 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/FetchBuffer.java: ## @@ -37,7 +36,7 @@ * * Note: this class is not thread-safe and is intended

[PR] KAFKA-15330: Add missing documentation of metrics introduced as part of KAFKA-15028 [kafka]

2023-10-03 Thread via GitHub
jolshan opened a new pull request, #14480: URL: https://github.com/apache/kafka/pull/14480 I've added details for VerificationFailureRate and VerificationTimeMs. I considered adding the documentation for the AddPartitionsToTxnVerification metrics, but I noticed that all the request m

Re: [PR] KAFKA-15526: Simplify the LogAppendInfo class [kafka]

2023-10-03 Thread via GitHub
splett2 commented on PR #14470: URL: https://github.com/apache/kafka/pull/14470#issuecomment-1745872570 thanks for the prompt reviews! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-03 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1344822614 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/FetchRequestManager.java: ## @@ -0,0 +1,186 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-03 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1344801112 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetchRequestManagerTest.java: ## @@ -0,0 +1,3565 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] KAFKA-15355: Message schema changes [kafka]

2023-10-03 Thread via GitHub
cmccabe commented on code in PR #14290: URL: https://github.com/apache/kafka/pull/14290#discussion_r1344800706 ## clients/src/main/resources/common/message/BrokerRegistrationRequest.json: ## @@ -53,6 +53,8 @@ { "name": "Rack", "type": "string", "versions": "0+", "nullableVe

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-03 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1344799794 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetchRequestManagerTest.java: ## @@ -0,0 +1,3565 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] KAFKA-15355: Message schema changes [kafka]

2023-10-03 Thread via GitHub
cmccabe commented on code in PR #14290: URL: https://github.com/apache/kafka/pull/14290#discussion_r1344799188 ## generator/src/main/java/org/apache/kafka/message/FieldSpec.java: ## @@ -157,11 +157,6 @@ private void checkTagInvariants() { this.taggedVersions

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-03 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1344795530 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetchRequestManagerTest.java: ## @@ -0,0 +1,3565 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] KAFKA-14593: fix gradle build [kafka]

2023-10-03 Thread via GitHub
bmscomp commented on PR #14474: URL: https://github.com/apache/kafka/pull/14474#issuecomment-1745787654 @raboof Thanks for the explanations -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-03 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1344786987 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetchRequestManagerTest.java: ## @@ -0,0 +1,3565 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-03 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1344786987 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetchRequestManagerTest.java: ## @@ -0,0 +1,3565 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-03 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1344780023 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetchRequestManagerTest.java: ## @@ -0,0 +1,3565 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] KAFKA-15355: Message schema changes [kafka]

2023-10-03 Thread via GitHub
rondagostino commented on code in PR #14290: URL: https://github.com/apache/kafka/pull/14290#discussion_r1344732697 ## clients/src/main/resources/common/message/BrokerRegistrationRequest.json: ## @@ -53,6 +53,8 @@ { "name": "Rack", "type": "string", "versions": "0+", "nulla

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-03 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1344762836 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java: ## @@ -106,7 +127,62 @@ public void onFailure(RuntimeException e) { return

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-03 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1344758105 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/PrototypeAsyncConsumer.java: ## @@ -192,71 +321,38 @@ public PrototypeAsyncConsumer(final Consumer

Re: [PR] KAFKA-14274 [6, 7]: Introduction of fetch request manager [kafka]

2023-10-03 Thread via GitHub
kirktrue commented on code in PR #14406: URL: https://github.com/apache/kafka/pull/14406#discussion_r1344756235 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/FetchRequestManager.java: ## @@ -0,0 +1,136 @@ +/* + * Licensed to the Apache Software Foundation

Re: [PR] KAFKA-14516: [1/N] Static Member leave request handling [kafka]

2023-10-03 Thread via GitHub
kirktrue commented on code in PR #14432: URL: https://github.com/apache/kafka/pull/14432#discussion_r1344754813 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -593,6 +609,9 @@ private void throwIfConsumerGroupHeartbeatReque

Re: [PR] MINOR: Add LEAVE_GROUP_EPOCH to GroupMetadataManager [kafka]

2023-10-03 Thread via GitHub
kirktrue commented on PR #14463: URL: https://github.com/apache/kafka/pull/14463#issuecomment-1745740977 @dajac @vamossagar12 Please take another look. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [PR] MINOR: Add LEAVE_GROUP_EPOCH to GroupMetadataManager [kafka]

2023-10-03 Thread via GitHub
kirktrue commented on code in PR #14463: URL: https://github.com/apache/kafka/pull/14463#discussion_r1344752797 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -131,6 +131,11 @@ */ public class GroupMetadataManager { +

Re: [PR] MINOR: Add LEAVE_GROUP_EPOCH to GroupMetadataManager [kafka]

2023-10-03 Thread via GitHub
kirktrue commented on PR #14463: URL: https://github.com/apache/kafka/pull/14463#issuecomment-1745740348 > Thanks for this PR @kirktrue ! I can see easily see how this can be extended for static membership case. I had just one question(LGTM o/w) => > > 1. Do [these](https://github.co

Re: [PR] KAFKA-14519; [1/N] Implement coordinator runtime metrics [kafka]

2023-10-03 Thread via GitHub
jolshan commented on code in PR #14417: URL: https://github.com/apache/kafka/pull/14417#discussion_r1344731581 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/CoordinatorRuntime.java: ## @@ -1242,7 +1307,8 @@ public void scheduleLoadOperation(

Re: [PR] KAFKA-14519; [1/N] Implement coordinator runtime metrics [kafka]

2023-10-03 Thread via GitHub
jolshan commented on code in PR #14417: URL: https://github.com/apache/kafka/pull/14417#discussion_r1344730913 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/runtime/CoordinatorRuntime.java: ## @@ -156,15 +165,16 @@ public CoordinatorRuntime build() {

Re: [PR] KAFKA-14519; [1/N] Implement coordinator runtime metrics [kafka]

2023-10-03 Thread via GitHub
jolshan commented on code in PR #14417: URL: https://github.com/apache/kafka/pull/14417#discussion_r1344717277 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/metrics/CoordinatorRuntimeMetrics.java: ## @@ -0,0 +1,100 @@ +/* + * Licensed to the Apache Softwa

Re: [PR] KAFKA-14519; [1/N] Implement coordinator runtime metrics [kafka]

2023-10-03 Thread via GitHub
jolshan commented on code in PR #14417: URL: https://github.com/apache/kafka/pull/14417#discussion_r1344716332 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinatorService.java: ## @@ -148,7 +157,9 @@ public GroupCoordinatorService build() {

Re: [PR] KAFKA-14519; [1/N] Implement coordinator runtime metrics [kafka]

2023-10-03 Thread via GitHub
jolshan commented on code in PR #14417: URL: https://github.com/apache/kafka/pull/14417#discussion_r1344714254 ## core/src/test/scala/unit/kafka/coordinator/group/CoordinatorLoaderImplTest.scala: ## @@ -283,7 +292,63 @@ class CoordinatorLoaderImplTest { minOneMessage =

[PR] Added LATEST_3_6 to the parameters [kafka]

2023-10-03 Thread via GitHub
apurushotama opened a new pull request, #14479: URL: https://github.com/apache/kafka/pull/14479 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of te

Re: [PR] Added LATEST_3_6 to the parameters [kafka]

2023-10-03 Thread via GitHub
apurushotama closed pull request #14479: Added LATEST_3_6 to the parameters URL: https://github.com/apache/kafka/pull/14479 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

Re: [PR] KAFKA-15526: Simplify the LogAppendInfo class [kafka]

2023-10-03 Thread via GitHub
jolshan commented on code in PR #14470: URL: https://github.com/apache/kafka/pull/14470#discussion_r1344697594 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -74,11 +74,20 @@ import scala.jdk.CollectionConverters._ /* * Result metadata of a log append operati

Re: [PR] KAFKA-15526: Simplify the LogAppendInfo class [kafka]

2023-10-03 Thread via GitHub
jolshan commented on code in PR #14470: URL: https://github.com/apache/kafka/pull/14470#discussion_r1344697202 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -763,20 +763,20 @@ class ReplicaManager(val config: KafkaConfig, val errorResults = (unverified

Re: [PR] KAFKA-15532: non active controllers return 0 for ZkWriteBeforelag [kafka]

2023-10-03 Thread via GitHub
mumrah commented on code in PR #14478: URL: https://github.com/apache/kafka/pull/14478#discussion_r1344679204 ## metadata/src/main/java/org/apache/kafka/controller/metrics/QuorumControllerMetrics.java: ## @@ -160,8 +160,8 @@ public Long value() { registry.ifPresent(

Re: [PR] KAFKA-15276: Implement partition assignment reconciliation [kafka]

2023-10-03 Thread via GitHub
lianetm commented on code in PR #14357: URL: https://github.com/apache/kafka/pull/14357#discussion_r1344659132 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AssignmentReconciler.java: ## @@ -85,122 +111,162 @@ enum ReconciliationResult { private final

Re: [PR] KAFKA-14912:Add a dynamic config for remote index cache size [kafka]

2023-10-03 Thread via GitHub
satishd commented on code in PR #14381: URL: https://github.com/apache/kafka/pull/14381#discussion_r1344604731 ## storage/src/main/java/org/apache/kafka/server/log/remote/storage/RemoteLogManagerConfig.java: ## @@ -197,7 +197,7 @@ public final class RemoteLogManagerConfig {

Re: [PR] MINOR: Add LEAVE_GROUP_EPOCH to GroupMetadataManager [kafka]

2023-10-03 Thread via GitHub
kirktrue commented on code in PR #14463: URL: https://github.com/apache/kafka/pull/14463#discussion_r1344606749 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupMetadataManager.java: ## @@ -131,6 +131,11 @@ */ public class GroupMetadataManager { +

Re: [PR] KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests [kafka]

2023-10-03 Thread via GitHub
philipnee commented on PR #14364: URL: https://github.com/apache/kafka/pull/14364#issuecomment-1745567545 @dajac @lianetm Thanks for the feedback. I addressed more of your comments. I wanted to point out that I filed 3 follow-up tickets to close some of the gaps. These are: - Propagate t

[jira] [Created] (KAFKA-15535) Add documentation of "remote.log.index.file.cache.total.size.bytes" configuration property.

2023-10-03 Thread Satish Duggana (Jira)
Satish Duggana created KAFKA-15535: -- Summary: Add documentation of "remote.log.index.file.cache.total.size.bytes" configuration property. Key: KAFKA-15535 URL: https://issues.apache.org/jira/browse/KAFKA-15535

Re: [PR] KAFKA-12989: MockClient should respect the request matcher passed to prepareUnsupportedVersionResponse [kafka]

2023-10-03 Thread via GitHub
kirktrue closed pull request #10980: KAFKA-12989: MockClient should respect the request matcher passed to prepareUnsupportedVersionResponse URL: https://github.com/apache/kafka/pull/10980 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests [kafka]

2023-10-03 Thread via GitHub
philipnee commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1344603478 ## clients/src/test/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManagerTest.java: ## @@ -58,7 +59,7 @@ public class HeartbeatRequestManagerTes

Re: [PR] KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests [kafka]

2023-10-03 Thread via GitHub
philipnee commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1344589907 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -0,0 +1,341 @@ +/* + * Licensed to the Apache Software Founda

[jira] [Created] (KAFKA-15534) Propagate client response time when timeout to the request handler

2023-10-03 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15534: -- Summary: Propagate client response time when timeout to the request handler Key: KAFKA-15534 URL: https://issues.apache.org/jira/browse/KAFKA-15534 Project: Kafka

Re: [PR] KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests [kafka]

2023-10-03 Thread via GitHub
lianetm commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r134456 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -0,0 +1,341 @@ +/* + * Licensed to the Apache Software Foundati

Re: [PR] KAFKA-15527: Add reverseRange and reverseAll query over kv-store in IQv2 [kafka]

2023-10-03 Thread via GitHub
hanyuzheng7 commented on PR #14477: URL: https://github.com/apache/kafka/pull/14477#issuecomment-1745523946 > Hey this is a public API change right? > > I think you need to update the kip to include the change and get it voted on > > I think this the range query kip https://cw

Re: [PR] KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests [kafka]

2023-10-03 Thread via GitHub
lianetm commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1344560113 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -0,0 +1,341 @@ +/* + * Licensed to the Apache Software Foundati

Re: [PR] KAFKA-14593: fix gradle build [kafka]

2023-10-03 Thread via GitHub
raboof commented on PR #14474: URL: https://github.com/apache/kafka/pull/14474#issuecomment-1745520847 at least `gradle dependencyCheckAggregate` fails for me: ``` > Configure project : Starting build with version 3.7.0-SNAPSHOT (commit id 5f676cce) using Gradle 8.3, Java 17 and

[PR] KAFKA-15532: non active controllers return 0 for ZkWriteBeforelag [kafka]

2023-10-03 Thread via GitHub
mannoopj opened a new pull request, #14478: URL: https://github.com/apache/kafka/pull/14478 Since only the active controller is performing the dual-write to ZK during a migration, it should be the only controller to report the ZkWriteBehindLag metric. Currently, if the cont

Re: [PR] KAFKA-14506: Implement DeleteGroups API and OffsetDelete API [kafka]

2023-10-03 Thread via GitHub
dajac commented on code in PR #14408: URL: https://github.com/apache/kafka/pull/14408#discussion_r1344554094 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinatorShard.java: ## @@ -262,6 +278,51 @@ public HeartbeatResponseData genericGroupHeartbe

Re: [PR] KAFKA-14506: Implement DeleteGroups API and OffsetDelete API [kafka]

2023-10-03 Thread via GitHub
dongnuo123 commented on code in PR #14408: URL: https://github.com/apache/kafka/pull/14408#discussion_r1344524966 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupCoordinatorShardTest.java: ## @@ -105,11 +119,126 @@ public void testCommitOffset() {

Re: [PR] KAFKA-14506: Implement DeleteGroups API and OffsetDelete API [kafka]

2023-10-03 Thread via GitHub
dongnuo123 commented on code in PR #14408: URL: https://github.com/apache/kafka/pull/14408#discussion_r1344524665 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupCoordinatorShardTest.java: ## @@ -105,11 +119,126 @@ public void testCommitOffset() {

Re: [PR] KAFKA-14506: Implement DeleteGroups API and OffsetDelete API [kafka]

2023-10-03 Thread via GitHub
dongnuo123 commented on code in PR #14408: URL: https://github.com/apache/kafka/pull/14408#discussion_r1344523685 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinatorService.java: ## @@ -259,30 +262,11 @@ public CompletableFuture consumerGroupH

Re: [PR] KAFKA-15415: On producer-batch retry, skip-backoff on a new leader [kafka]

2023-10-03 Thread via GitHub
kirktrue commented on code in PR #14384: URL: https://github.com/apache/kafka/pull/14384#discussion_r1344502781 ## clients/src/main/java/org/apache/kafka/clients/producer/internals/ProducerBatch.java: ## @@ -94,9 +100,45 @@ public ProducerBatch(TopicPartition tp, MemoryRecordsB

Re: [PR] KAFKA-15527: Add reverseRange and reverseAll query over kv-store in IQv2 [kafka]

2023-10-03 Thread via GitHub
wcarlson5 commented on PR #14477: URL: https://github.com/apache/kafka/pull/14477#issuecomment-1745466006 Hey this is a public API change right? I think you need to update the kip to include the change and get it voted on I think this the range query kip https://cwiki.apache.o

Re: [PR] KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests [kafka]

2023-10-03 Thread via GitHub
philipnee commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1344515152 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -0,0 +1,330 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests [kafka]

2023-10-03 Thread via GitHub
philipnee commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1344512748 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -0,0 +1,324 @@ +/* + * Licensed to the Apache Software Founda

[jira] [Created] (KAFKA-15533) Ensure HeartbeatRequestManager only send out some fields once

2023-10-03 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15533: -- Summary: Ensure HeartbeatRequestManager only send out some fields once Key: KAFKA-15533 URL: https://issues.apache.org/jira/browse/KAFKA-15533 Project: Kafka Is

Re: [PR] KAFKA-14506: Implement DeleteGroups API and OffsetDelete API [kafka]

2023-10-03 Thread via GitHub
dongnuo123 commented on code in PR #14408: URL: https://github.com/apache/kafka/pull/14408#discussion_r1344502310 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupCoordinatorShardTest.java: ## @@ -105,11 +119,126 @@ public void testCommitOffset() {

Re: [PR] KAFKA-14506: Implement DeleteGroups API and OffsetDelete API [kafka]

2023-10-03 Thread via GitHub
dongnuo123 commented on code in PR #14408: URL: https://github.com/apache/kafka/pull/14408#discussion_r1344498860 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinatorShard.java: ## @@ -262,6 +278,51 @@ public HeartbeatResponseData genericGroupHe

Re: [PR] KAFKA-14506: Implement DeleteGroups API and OffsetDelete API [kafka]

2023-10-03 Thread via GitHub
dongnuo123 commented on code in PR #14408: URL: https://github.com/apache/kafka/pull/14408#discussion_r1344498860 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinatorShard.java: ## @@ -262,6 +278,51 @@ public HeartbeatResponseData genericGroupHe

Re: [PR] KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests [kafka]

2023-10-03 Thread via GitHub
philipnee commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1344497810 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -0,0 +1,294 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] KAFKA-14519; [1/N] Implement coordinator runtime metrics [kafka]

2023-10-03 Thread via GitHub
dajac commented on code in PR #14417: URL: https://github.com/apache/kafka/pull/14417#discussion_r1341657675 ## core/src/test/scala/unit/kafka/coordinator/group/CoordinatorLoaderImplTest.scala: ## @@ -283,7 +292,63 @@ class CoordinatorLoaderImplTest { minOneMessage = tr

Re: [PR] KAFKA-15276: Implement partition assignment reconciliation [kafka]

2023-10-03 Thread via GitHub
lianetm commented on code in PR #14357: URL: https://github.com/apache/kafka/pull/14357#discussion_r1344472501 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AssignmentReconciler.java: ## @@ -0,0 +1,355 @@ +/* + * Licensed to the Apache Software Foundation

[jira] [Created] (KAFKA-15532) ZkWriteBehindLag should not be reported by inactive controllers

2023-10-03 Thread David Arthur (Jira)
David Arthur created KAFKA-15532: Summary: ZkWriteBehindLag should not be reported by inactive controllers Key: KAFKA-15532 URL: https://issues.apache.org/jira/browse/KAFKA-15532 Project: Kafka

Re: [PR] KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests [kafka]

2023-10-03 Thread via GitHub
philipnee commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1344462484 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -0,0 +1,330 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] KAFKA-15526: Simplify the LogAppendInfo class [kafka]

2023-10-03 Thread via GitHub
splett2 commented on code in PR #14470: URL: https://github.com/apache/kafka/pull/14470#discussion_r1344456909 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -74,11 +74,20 @@ import scala.jdk.CollectionConverters._ /* * Result metadata of a log append operati

Re: [PR] KAFKA-15526: Simplify the LogAppendInfo class [kafka]

2023-10-03 Thread via GitHub
splett2 commented on code in PR #14470: URL: https://github.com/apache/kafka/pull/14470#discussion_r131966 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -766,7 +775,8 @@ class ReplicaManager(val config: KafkaConfig, } topicPartitio

[jira] [Created] (KAFKA-15531) Ensure coordinator node is removed upon disconnection exception

2023-10-03 Thread Philip Nee (Jira)
Philip Nee created KAFKA-15531: -- Summary: Ensure coordinator node is removed upon disconnection exception Key: KAFKA-15531 URL: https://issues.apache.org/jira/browse/KAFKA-15531 Project: Kafka

Re: [PR] KAFKA-15276: Implement partition assignment reconciliation [kafka]

2023-10-03 Thread via GitHub
lianetm commented on code in PR #14357: URL: https://github.com/apache/kafka/pull/14357#discussion_r1344406010 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AssignmentReconciler.java: ## @@ -44,21 +45,46 @@ import java.util.stream.Collectors; /** - * {

Re: [PR] KAFKA-15526: Simplify the LogAppendInfo class [kafka]

2023-10-03 Thread via GitHub
jolshan commented on code in PR #14470: URL: https://github.com/apache/kafka/pull/14470#discussion_r133473 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -766,7 +775,8 @@ class ReplicaManager(val config: KafkaConfig, } topicPartitio

Re: [PR] KAFKA-15526: Simplify the LogAppendInfo class [kafka]

2023-10-03 Thread via GitHub
splett2 commented on code in PR #14470: URL: https://github.com/apache/kafka/pull/14470#discussion_r131966 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -766,7 +775,8 @@ class ReplicaManager(val config: KafkaConfig, } topicPartitio

[jira] [Commented] (KAFKA-15028) AddPartitionsToTxnManager metrics

2023-10-03 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17771560#comment-17771560 ] Justine Olshan commented on KAFKA-15028: Thanks [~divijvaidya] I will continue w

[jira] [Updated] (KAFKA-15530) Add missing documentation of metrics introduced as part of KAFKA-15028

2023-10-03 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan updated KAFKA-15530: --- Summary: Add missing documentation of metrics introduced as part of KAFKA-15028 (was: Add m

[jira] [Assigned] (KAFKA-15530) Add missing documentation of metrics introduced as part of KAFKA-15196

2023-10-03 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan reassigned KAFKA-15530: -- Assignee: Justine Olshan > Add missing documentation of metrics introduced as part of

Re: [PR] KAFKA-14595 Move ReassignPartitionsCommand to java [kafka]

2023-10-03 Thread via GitHub
nizhikov commented on PR #13247: URL: https://github.com/apache/kafka/pull/13247#issuecomment-1745360537 @jolshan Please, join the final review to rewrite `ReassignPartitionsCommand` to java if you have spare cycles :) -- This is an automated message from the Apache Git Service. To respon

Re: [PR] KAFKA-14595 ReassignPartitionsIntegrationTest rewritten in java [kafka]

2023-10-03 Thread via GitHub
jolshan commented on code in PR #14456: URL: https://github.com/apache/kafka/pull/14456#discussion_r1344425267 ## build.gradle: ## @@ -1908,6 +1908,7 @@ project(':tools') { testImplementation project(':connect:api') testImplementation project(':connect:runtime') t

Re: [PR] KAFKA-15526: Simplify the LogAppendInfo class [kafka]

2023-10-03 Thread via GitHub
jolshan commented on code in PR #14470: URL: https://github.com/apache/kafka/pull/14470#discussion_r1344423738 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -766,7 +775,8 @@ class ReplicaManager(val config: KafkaConfig, } topicPartitio

Re: [PR] [MINOR] Fix compilation [kafka]

2023-10-03 Thread via GitHub
fvaleri commented on PR #14475: URL: https://github.com/apache/kafka/pull/14475#issuecomment-1745353991 Thanks @nizhikov. Works fine. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] KAFKA-15526: Simplify the LogAppendInfo class [kafka]

2023-10-03 Thread via GitHub
jolshan commented on code in PR #14470: URL: https://github.com/apache/kafka/pull/14470#discussion_r1344419654 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -766,7 +775,8 @@ class ReplicaManager(val config: KafkaConfig, } topicPartitio

Re: [PR] KAFKA-15526: Simplify the LogAppendInfo class [kafka]

2023-10-03 Thread via GitHub
jolshan commented on code in PR #14470: URL: https://github.com/apache/kafka/pull/14470#discussion_r1344414660 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -74,11 +74,20 @@ import scala.jdk.CollectionConverters._ /* * Result metadata of a log append operati

[PR] KAFKA-15527: Add reverseRange and reverseAll query over kv-store in IQv2 [kafka]

2023-10-03 Thread via GitHub
hanyuzheng7 opened a new pull request, #14477: URL: https://github.com/apache/kafka/pull/14477 ### Summary Update an implementation of the Query interface, introduced in [KIP-796: Interactive Query v2](https://cwiki.apache.org/confluence/display/KAFKA/KIP-796%3A+Interactive+Query+v2) ,

Re: [PR] KAFKA-15526: Simplify the LogAppendInfo class [kafka]

2023-10-03 Thread via GitHub
jolshan commented on code in PR #14470: URL: https://github.com/apache/kafka/pull/14470#discussion_r1344411599 ## core/src/main/scala/kafka/log/UnifiedLog.scala: ## @@ -892,16 +889,11 @@ class UnifiedLog(@volatile var logStartOffset: Long, maybeDuplicate match {

[jira] [Updated] (KAFKA-15527) Add reverseRange and reverseAll query over kv-store in IQv2

2023-10-03 Thread Hanyu Zheng (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hanyu Zheng updated KAFKA-15527: Description: Add reverseRange and reverseAll query over kv-store in IQv2 Update an implementation

Re: [PR] KAFKA-15276: Implement partition assignment reconciliation [kafka]

2023-10-03 Thread via GitHub
lianetm commented on code in PR #14357: URL: https://github.com/apache/kafka/pull/14357#discussion_r1344406010 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AssignmentReconciler.java: ## @@ -44,21 +45,46 @@ import java.util.stream.Collectors; /** - * {

Re: [PR] KAFKA-15415 On producer-batch retry, skip-backoff on a new leader [kafka]

2023-10-03 Thread via GitHub
wcarlson5 commented on code in PR #14384: URL: https://github.com/apache/kafka/pull/14384#discussion_r1344314253 ## clients/src/main/java/org/apache/kafka/clients/producer/internals/ProducerBatch.java: ## @@ -94,9 +100,45 @@ public ProducerBatch(TopicPartition tp, MemoryRecords

Re: [PR] KAFKA-14595 ReassignPartitionsIntegrationTest rewritten in java [kafka]

2023-10-03 Thread via GitHub
nizhikov commented on code in PR #14456: URL: https://github.com/apache/kafka/pull/14456#discussion_r1344361140 ## build.gradle: ## @@ -1908,6 +1908,7 @@ project(':tools') { testImplementation project(':connect:api') testImplementation project(':connect:runtime')

Re: [PR] [MINOR] Fix compilation [kafka]

2023-10-03 Thread via GitHub
mimaison merged PR #14475: URL: https://github.com/apache/kafka/pull/14475 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache

[PR] MINOR: fix dependencycheck warnings [kafka]

2023-10-03 Thread via GitHub
raboof opened a new pull request, #14476: URL: https://github.com/apache/kafka/pull/14476 Add suppressions and skip benchmarking/testing projects ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build

Re: [PR] KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests [kafka]

2023-10-03 Thread via GitHub
philipnee commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1344336463 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -0,0 +1,330 @@ +/* + * Licensed to the Apache Software Founda

Re: [PR] KAFKA-14593: Move LeaderElectionCommand to tools [kafka]

2023-10-03 Thread via GitHub
nizhikov commented on PR #13204: URL: https://github.com/apache/kafka/pull/13204#issuecomment-1745233460 Hello @OmniaGM @mimaison @fvaleri Looks like https://github.com/apache/kafka/commit/7553d3f562f3af6c7f9b062b9220bcad80b00478 overlaps with https://github.com/apache/kafka/commit/

Re: [PR] KAFKA-14595 ReassignPartitionsIntegrationTest rewritten in java [kafka]

2023-10-03 Thread via GitHub
nizhikov commented on code in PR #14456: URL: https://github.com/apache/kafka/pull/14456#discussion_r1344317637 ## build.gradle: ## @@ -1908,6 +1908,7 @@ project(':tools') { testImplementation project(':connect:api') testImplementation project(':connect:runtime')

Re: [PR] [MINOR] Fix compilation [kafka]

2023-10-03 Thread via GitHub
nizhikov commented on PR #14475: URL: https://github.com/apache/kafka/pull/14475#issuecomment-1745230376 Hello @OmniaGM @mimaison @fvaleri Looks like 7553d3f562f3af6c7f9b062b9220bcad80b00478 overlaps with 8f8dbad564ffd9be409bb85edadbc40659cd0a56 and break trunk compilation. Th

[PR] [MINOR] Fix compilation [kafka]

2023-10-03 Thread via GitHub
nizhikov opened a new pull request, #14475: URL: https://github.com/apache/kafka/pull/14475 Fix trunk compilation after 7553d3f562f3af6c7f9b062b9220bcad80b00478 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test covera

  1   2   >