[GitHub] [kafka] lihaosky commented on a diff in pull request #14097: KAFKA-15022: [4/N] use client tag assignor for rack aware standby task assignment

2023-07-27 Thread via GitHub
lihaosky commented on code in PR #14097: URL: https://github.com/apache/kafka/pull/14097#discussion_r1277119690 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/assignment/RackAwareTaskAssignor.java: ## @@ -63,11 +62,11 @@ public RackAwareTaskAssignor(final

[GitHub] [kafka] satishd merged pull request #14049: KAFKA-14038: Optimise calculation of size for log in remote tier

2023-07-27 Thread via GitHub
satishd merged PR #14049: URL: https://github.com/apache/kafka/pull/14049 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] satishd commented on pull request #14049: KAFKA-14038: Optimise calculation of size for log in remote tier

2023-07-27 Thread via GitHub
satishd commented on PR #14049: URL: https://github.com/apache/kafka/pull/14049#issuecomment-1655051965 A few unrelated test failures in Jenkins jobs, merging it to trunk. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[jira] [Comment Edited] (KAFKA-15231) Add ability to pause/resume Remote Log Manager tasks

2023-07-27 Thread Satish Duggana (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17748406#comment-17748406 ] Satish Duggana edited comment on KAFKA-15231 at 7/28/23 4:48 AM: - There

[jira] [Commented] (KAFKA-15231) Add ability to pause/resume Remote Log Manager tasks

2023-07-27 Thread Satish Duggana (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17748406#comment-17748406 ] Satish Duggana commented on KAFKA-15231: There can be scenarios where you want to control more

[jira] [Updated] (KAFKA-15265) Remote copy/fetch quotas for tiered storage.

2023-07-27 Thread Satish Duggana (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Satish Duggana updated KAFKA-15265: --- Description: Related KIP:

[jira] [Created] (KAFKA-15265) Remote copy/fetch quotas for tiered storage.

2023-07-27 Thread Satish Duggana (Jira)
Satish Duggana created KAFKA-15265: -- Summary: Remote copy/fetch quotas for tiered storage. Key: KAFKA-15265 URL: https://issues.apache.org/jira/browse/KAFKA-15265 Project: Kafka Issue Type:

[jira] [Commented] (KAFKA-15214) Add metrics for OffsetOutOfRangeException when tiered storage is enabled

2023-07-27 Thread Satish Duggana (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17748402#comment-17748402 ] Satish Duggana commented on KAFKA-15214: [~yaolixin] Is the client trying to fetch the remote

[GitHub] [kafka] github-actions[bot] commented on pull request #13568: KAFKA-14906:Extract the coordinator service log from server log

2023-07-27 Thread via GitHub
github-actions[bot] commented on PR #13568: URL: https://github.com/apache/kafka/pull/13568#issuecomment-1654936322 This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please ask a committer for review. If the PR has

[GitHub] [kafka] mjsax commented on a diff in pull request #14097: KAFKA-15022: [4/N] use client tag assignor for rack aware standby task assignment

2023-07-27 Thread via GitHub
mjsax commented on code in PR #14097: URL: https://github.com/apache/kafka/pull/14097#discussion_r1277018411 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/assignment/ClientTagAwareStandbyTaskAssignorTest.java: ## @@ -313,20 +315,145 @@ public void

[GitHub] [kafka] mjsax commented on a diff in pull request #14097: KAFKA-15022: [4/N] use client tag assignor for rack aware standby task assignment

2023-07-27 Thread via GitHub
mjsax commented on code in PR #14097: URL: https://github.com/apache/kafka/pull/14097#discussion_r1277018411 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/assignment/ClientTagAwareStandbyTaskAssignorTest.java: ## @@ -313,20 +315,145 @@ public void

[GitHub] [kafka] mjsax commented on a diff in pull request #14097: KAFKA-15022: [4/N] use client tag assignor for rack aware standby task assignment

2023-07-27 Thread via GitHub
mjsax commented on code in PR #14097: URL: https://github.com/apache/kafka/pull/14097#discussion_r1277018411 ## streams/src/test/java/org/apache/kafka/streams/processor/internals/assignment/ClientTagAwareStandbyTaskAssignorTest.java: ## @@ -313,20 +315,145 @@ public void

[GitHub] [kafka] mjsax commented on a diff in pull request #14097: KAFKA-15022: [4/N] use client tag assignor for rack aware standby task assignment

2023-07-27 Thread via GitHub
mjsax commented on code in PR #14097: URL: https://github.com/apache/kafka/pull/14097#discussion_r1277015051 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/assignment/RackAwareTaskAssignor.java: ## @@ -63,11 +62,11 @@ public RackAwareTaskAssignor(final

[GitHub] [kafka] mjsax commented on a diff in pull request #14097: KAFKA-15022: [4/N] use client tag assignor for rack aware standby task assignment

2023-07-27 Thread via GitHub
mjsax commented on code in PR #14097: URL: https://github.com/apache/kafka/pull/14097#discussion_r1277015051 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/assignment/RackAwareTaskAssignor.java: ## @@ -63,11 +62,11 @@ public RackAwareTaskAssignor(final

[GitHub] [kafka] mjsax commented on a diff in pull request #14097: KAFKA-15022: [4/N] use client tag assignor for rack aware standby task assignment

2023-07-27 Thread via GitHub
mjsax commented on code in PR #14097: URL: https://github.com/apache/kafka/pull/14097#discussion_r1277015051 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/assignment/RackAwareTaskAssignor.java: ## @@ -63,11 +62,11 @@ public RackAwareTaskAssignor(final

[GitHub] [kafka] mjsax commented on a diff in pull request #14097: KAFKA-15022: [4/N] use client tag assignor for rack aware standby task assignment

2023-07-27 Thread via GitHub
mjsax commented on code in PR #14097: URL: https://github.com/apache/kafka/pull/14097#discussion_r1277014726 ## streams/src/main/java/org/apache/kafka/streams/processor/internals/assignment/RackAwareTaskAssignor.java: ## @@ -63,11 +62,11 @@ public RackAwareTaskAssignor(final

[jira] [Updated] (KAFKA-15264) Compared with 1.1.0zk, the peak throughput of 3.5.1kraft is very jitter

2023-07-27 Thread jianbin.chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jianbin.chen updated KAFKA-15264: - Attachment: image-2023-07-28-09-52-38-941.png Description: I was preparing to upgrade from

[jira] [Commented] (KAFKA-15259) Kafka Streams does not continue processing due to rollback despite ProductionExceptionHandlerResponse.CONTINUE if using execute_once

2023-07-27 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17748367#comment-17748367 ] Matthias J. Sax commented on KAFKA-15259: - Thanks for getting back – I was just asking a general

[jira] [Updated] (KAFKA-15264) Compared with 1.1.0zk, the peak throughput of 3.5.1kraft is very jitter

2023-07-27 Thread jianbin.chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jianbin.chen updated KAFKA-15264: - Attachment: image-2023-07-28-09-51-01-662.png Description: I was preparing to upgrade from

[GitHub] [kafka] jeffkbkim commented on pull request #14117: MINOR: Code cleanups in group-coordinator module

2023-07-27 Thread via GitHub
jeffkbkim commented on PR #14117: URL: https://github.com/apache/kafka/pull/14117#issuecomment-1654851021 can you point me to where the log context includes the topic partition info? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[jira] [Created] (KAFKA-15264) Compared with 1.1.0zk, the peak throughput of 3.5.1kraft is very jitter

2023-07-27 Thread jianbin.chen (Jira)
jianbin.chen created KAFKA-15264: Summary: Compared with 1.1.0zk, the peak throughput of 3.5.1kraft is very jitter Key: KAFKA-15264 URL: https://issues.apache.org/jira/browse/KAFKA-15264 Project:

[GitHub] [kafka] philipnee commented on pull request #14086: MINOR: Test assign() and assignment() in the integration test

2023-07-27 Thread via GitHub
philipnee commented on PR #14086: URL: https://github.com/apache/kafka/pull/14086#issuecomment-1654826392 Hey @junrao Thanks a lot for reviewing this. I fixed the broken build and here are the failing tests. I believe they are unrelated: ``` Build / JDK 17 and Scala 2.13 /

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #14056: KAFKA-14501: Implement Heartbeat protocol in new GroupCoordinator

2023-07-27 Thread via GitHub
jeffkbkim commented on code in PR #14056: URL: https://github.com/apache/kafka/pull/14056#discussion_r1276971308 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -8540,5 +8568,1182 @@ private static class RebalanceResult

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #14056: KAFKA-14501: Implement Heartbeat protocol in new GroupCoordinator

2023-07-27 Thread via GitHub
jeffkbkim commented on code in PR #14056: URL: https://github.com/apache/kafka/pull/14056#discussion_r1276959521 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -8540,5 +8568,1182 @@ private static class RebalanceResult

[GitHub] [kafka] cmccabe merged pull request #13643: KAFKA-15213: provide the exact offset to QuorumController.replay

2023-07-27 Thread via GitHub
cmccabe merged PR #13643: URL: https://github.com/apache/kafka/pull/13643 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #14056: KAFKA-14501: Implement Heartbeat protocol in new GroupCoordinator

2023-07-27 Thread via GitHub
jeffkbkim commented on code in PR #14056: URL: https://github.com/apache/kafka/pull/14056#discussion_r1276944101 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -8540,5 +8568,1182 @@ private static class RebalanceResult

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #14056: KAFKA-14501: Implement Heartbeat protocol in new GroupCoordinator

2023-07-27 Thread via GitHub
jeffkbkim commented on code in PR #14056: URL: https://github.com/apache/kafka/pull/14056#discussion_r1276943880 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -8540,5 +8568,1182 @@ private static class RebalanceResult

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #14056: KAFKA-14501: Implement Heartbeat protocol in new GroupCoordinator

2023-07-27 Thread via GitHub
jeffkbkim commented on code in PR #14056: URL: https://github.com/apache/kafka/pull/14056#discussion_r1276942521 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -8540,5 +8568,1182 @@ private static class RebalanceResult

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #14056: KAFKA-14501: Implement Heartbeat protocol in new GroupCoordinator

2023-07-27 Thread via GitHub
jeffkbkim commented on code in PR #14056: URL: https://github.com/apache/kafka/pull/14056#discussion_r1276939557 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -8540,5 +8568,1182 @@ private static class RebalanceResult

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #14056: KAFKA-14501: Implement Heartbeat protocol in new GroupCoordinator

2023-07-27 Thread via GitHub
jeffkbkim commented on code in PR #14056: URL: https://github.com/apache/kafka/pull/14056#discussion_r1276936889 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -8540,5 +8568,1182 @@ private static class RebalanceResult

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #14056: KAFKA-14501: Implement Heartbeat protocol in new GroupCoordinator

2023-07-27 Thread via GitHub
jeffkbkim commented on code in PR #14056: URL: https://github.com/apache/kafka/pull/14056#discussion_r1276923811 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -8540,5 +8568,1182 @@ private static class RebalanceResult

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #14056: KAFKA-14501: Implement Heartbeat protocol in new GroupCoordinator

2023-07-27 Thread via GitHub
jeffkbkim commented on code in PR #14056: URL: https://github.com/apache/kafka/pull/14056#discussion_r1276922896 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -8540,5 +8568,1182 @@ private static class RebalanceResult

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #14056: KAFKA-14501: Implement Heartbeat protocol in new GroupCoordinator

2023-07-27 Thread via GitHub
jeffkbkim commented on code in PR #14056: URL: https://github.com/apache/kafka/pull/14056#discussion_r1276921862 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -8540,5 +8568,1182 @@ private static class RebalanceResult

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #14056: KAFKA-14501: Implement Heartbeat protocol in new GroupCoordinator

2023-07-27 Thread via GitHub
jeffkbkim commented on code in PR #14056: URL: https://github.com/apache/kafka/pull/14056#discussion_r1276919519 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -8540,5 +8568,1182 @@ private static class RebalanceResult

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #14056: KAFKA-14501: Implement Heartbeat protocol in new GroupCoordinator

2023-07-27 Thread via GitHub
jeffkbkim commented on code in PR #14056: URL: https://github.com/apache/kafka/pull/14056#discussion_r1276919382 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -8540,5 +8568,1182 @@ private static class RebalanceResult

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #14056: KAFKA-14501: Implement Heartbeat protocol in new GroupCoordinator

2023-07-27 Thread via GitHub
jeffkbkim commented on code in PR #14056: URL: https://github.com/apache/kafka/pull/14056#discussion_r1276874868 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/GroupCoordinatorService.java: ## @@ -368,9 +369,29 @@ public CompletableFuture heartbeat(

[GitHub] [kafka] philipnee opened a new pull request, #14118: KAFKA-14875: Implement wakeup

2023-07-27 Thread via GitHub
philipnee opened a new pull request, #14118: URL: https://github.com/apache/kafka/pull/14118 Continuation of https://github.com/apache/kafka/pull/13490. I closed the original one due to rebase difficulties. **Summary Implemented wakeup() mechanism using a WakeupTrigger class to

[GitHub] [kafka] philipnee closed pull request #13490: KAFKA-14875: Implement wakeup

2023-07-27 Thread via GitHub
philipnee closed pull request #13490: KAFKA-14875: Implement wakeup URL: https://github.com/apache/kafka/pull/13490 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #14099: KAFKA-14702: Extend server side assignor to support rack aware replica placement

2023-07-27 Thread via GitHub
rreddy-22 commented on code in PR #14099: URL: https://github.com/apache/kafka/pull/14099#discussion_r1276830477 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/RecordHelpersTest.java: ## @@ -764,4 +813,39 @@ public void

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #14099: KAFKA-14702: Extend server side assignor to support rack aware replica placement

2023-07-27 Thread via GitHub
rreddy-22 commented on code in PR #14099: URL: https://github.com/apache/kafka/pull/14099#discussion_r1276830285 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -5567,7 +5600,7 @@ public void

[GitHub] [kafka] rondagostino commented on a diff in pull request #14084: [MINOR] Add latest versions to kraft upgrade kafkatest

2023-07-27 Thread via GitHub
rondagostino commented on code in PR #14084: URL: https://github.com/apache/kafka/pull/14084#discussion_r1276771939 ## tests/kafkatest/version.py: ## @@ -252,3 +252,7 @@ def get_version(node=None): V_3_5_0 = KafkaVersion("3.5.0") V_3_5_1 = KafkaVersion("3.5.1") LATEST_3_5 =

[jira] [Assigned] (KAFKA-14976) Left/outer stream-stream joins create KV stores that aren't customizable

2023-07-27 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman reassigned KAFKA-14976: -- Assignee: Almog Gavra > Left/outer stream-stream joins create KV

[jira] [Updated] (KAFKA-14976) Left/outer stream-stream joins create KV stores that aren't customizable

2023-07-27 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman updated KAFKA-14976: --- Labels: kip (was: needs-kip) > Left/outer stream-stream joins create KV

[jira] [Commented] (KAFKA-14976) Left/outer stream-stream joins create KV stores that aren't customizable

2023-07-27 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17748314#comment-17748314 ] A. Sophie Blee-Goldman commented on KAFKA-14976: Will be addressed by

[GitHub] [kafka] ableegoldman commented on pull request #13920: KAFKA-15106 fix AbstractStickyAssignor isBalanced predict

2023-07-27 Thread via GitHub
ableegoldman commented on PR #13920: URL: https://github.com/apache/kafka/pull/13920#issuecomment-1654436865 @flashmouse you can decrease the consumer count. For one thing, this covers the edge case where the consumers do not all have the same topic subscription, which seems to be quite

[GitHub] [kafka] dajac commented on a diff in pull request #14099: KAFKA-14702: Extend server side assignor to support rack aware replica placement

2023-07-27 Thread via GitHub
dajac commented on code in PR #14099: URL: https://github.com/apache/kafka/pull/14099#discussion_r1276742165 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -5567,7 +5600,7 @@ public void

[GitHub] [kafka] dajac commented on a diff in pull request #14099: KAFKA-14702: Extend server side assignor to support rack aware replica placement

2023-07-27 Thread via GitHub
dajac commented on code in PR #14099: URL: https://github.com/apache/kafka/pull/14099#discussion_r1276739854 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/RecordHelpersTest.java: ## @@ -764,4 +813,39 @@ public void testNewOffsetCommitTombstoneRecord() {

[GitHub] [kafka] dajac opened a new pull request, #14117: MINOR: Code cleanups in group-coordinator module

2023-07-27 Thread via GitHub
dajac opened a new pull request, #14117: URL: https://github.com/apache/kafka/pull/14117 This patch does a few code cleanups in the group-coordinator module. - It renames `Coordinator` to `CoordinatorShard`; - It renames `ReplicatedGroupCoordinator` to `GroupCoordinatorShard`. I was

[GitHub] [kafka] kamalcph opened a new pull request, #14116: KAFKA-15167: Tiered Storage Test Harness Framework

2023-07-27 Thread via GitHub
kamalcph opened a new pull request, #14116: URL: https://github.com/apache/kafka/pull/14116 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #14099: KAFKA-14702: Extend server side assignor to support rack aware replica placement

2023-07-27 Thread via GitHub
rreddy-22 commented on code in PR #14099: URL: https://github.com/apache/kafka/pull/14099#discussion_r1276710063 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/RecordHelpersTest.java: ## @@ -764,4 +813,39 @@ public void

[GitHub] [kafka] cmccabe commented on pull request #13643: KAFKA-15213: provide the exact offset to QuorumController.replay

2023-07-27 Thread via GitHub
cmccabe commented on PR #13643: URL: https://github.com/apache/kafka/pull/13643#issuecomment-1654254344 > I couldn't find a test for the new KafkaRaftClient::scheduleAtomicAppend. I added `KafkaRaftClientTest.testAppendWithRequiredBaseOffset`. -- This is an automated message from

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #14099: KAFKA-14702: Extend server side assignor to support rack aware replica placement

2023-07-27 Thread via GitHub
rreddy-22 commented on code in PR #14099: URL: https://github.com/apache/kafka/pull/14099#discussion_r1276706144 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -5692,12 +5725,60 @@ private void

[GitHub] [kafka] cmccabe commented on pull request #13643: KAFKA-15213: provide the exact offset to QuorumController.replay

2023-07-27 Thread via GitHub
cmccabe commented on PR #13643: URL: https://github.com/apache/kafka/pull/13643#issuecomment-1654251894 > @cmccabe, I don't follow this comment. When the client calls KafkaRaftClient::schedule{Atomic}Append the KafkaRaftClient compare the provided offset with the nextOffset stored in the

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #14099: KAFKA-14702: Extend server side assignor to support rack aware replica placement

2023-07-27 Thread via GitHub
rreddy-22 commented on code in PR #14099: URL: https://github.com/apache/kafka/pull/14099#discussion_r1276701201 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -5567,7 +5600,7 @@ public void

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #14099: KAFKA-14702: Extend server side assignor to support rack aware replica placement

2023-07-27 Thread via GitHub
rreddy-22 commented on code in PR #14099: URL: https://github.com/apache/kafka/pull/14099#discussion_r1276693242 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/TopicMetadata.java: ## @@ -40,23 +45,31 @@ public class TopicMetadata { */

[GitHub] [kafka] cmccabe commented on a diff in pull request #13643: KAFKA-15213: provide the exact offset to QuorumController.replay

2023-07-27 Thread via GitHub
cmccabe commented on code in PR #13643: URL: https://github.com/apache/kafka/pull/13643#discussion_r1276690892 ## raft/src/main/java/org/apache/kafka/raft/RaftClient.java: ## @@ -171,16 +172,21 @@ default void beginShutdown() {} * to resign its leadership. The state

[GitHub] [kafka] dajac commented on a diff in pull request #14099: KAFKA-14702: Extend server side assignor to support rack aware replica placement

2023-07-27 Thread via GitHub
dajac commented on code in PR #14099: URL: https://github.com/apache/kafka/pull/14099#discussion_r1276677238 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/consumer/ConsumerGroupTest.java: ## @@ -440,12 +442,13 @@ public void

[GitHub] [kafka] dajac commented on a diff in pull request #14099: KAFKA-14702: Extend server side assignor to support rack aware replica placement

2023-07-27 Thread via GitHub
dajac commented on code in PR #14099: URL: https://github.com/apache/kafka/pull/14099#discussion_r1276671418 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/consumer/TargetAssignmentBuilderTest.java: ## @@ -84,14 +85,15 @@ public void addGroupMember(

[GitHub] [kafka] dajac commented on a diff in pull request #14099: KAFKA-14702: Extend server side assignor to support rack aware replica placement

2023-07-27 Thread via GitHub
dajac commented on code in PR #14099: URL: https://github.com/apache/kafka/pull/14099#discussion_r1276670928 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/consumer/SubscribedTopicMetadataTest.java: ## @@ -0,0 +1,74 @@ +/* + * Licensed to the Apache

[GitHub] [kafka] dajac commented on a diff in pull request #14099: KAFKA-14702: Extend server side assignor to support rack aware replica placement

2023-07-27 Thread via GitHub
dajac commented on code in PR #14099: URL: https://github.com/apache/kafka/pull/14099#discussion_r1276670286 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/assignor/RangeAssignorTest.java: ## @@ -571,4 +716,14 @@ private void assertAssignment(Map>>

[GitHub] [kafka] dajac commented on a diff in pull request #14099: KAFKA-14702: Extend server side assignor to support rack aware replica placement

2023-07-27 Thread via GitHub
dajac commented on code in PR #14099: URL: https://github.com/apache/kafka/pull/14099#discussion_r1276669652 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/RecordHelpersTest.java: ## @@ -764,4 +813,39 @@ public void testNewOffsetCommitTombstoneRecord() {

[GitHub] [kafka] dajac commented on a diff in pull request #14099: KAFKA-14702: Extend server side assignor to support rack aware replica placement

2023-07-27 Thread via GitHub
dajac commented on code in PR #14099: URL: https://github.com/apache/kafka/pull/14099#discussion_r1276668936 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/RecordHelpersTest.java: ## @@ -612,15 +616,14 @@ public void

[GitHub] [kafka] dajac commented on a diff in pull request #14099: KAFKA-14702: Extend server side assignor to support rack aware replica placement

2023-07-27 Thread via GitHub
dajac commented on code in PR #14099: URL: https://github.com/apache/kafka/pull/14099#discussion_r127851 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -5692,12 +5725,60 @@ private void

[GitHub] [kafka] dajac commented on a diff in pull request #14099: KAFKA-14702: Extend server side assignor to support rack aware replica placement

2023-07-27 Thread via GitHub
dajac commented on code in PR #14099: URL: https://github.com/apache/kafka/pull/14099#discussion_r1276663083 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -5692,12 +5725,60 @@ private void

[GitHub] [kafka] dajac commented on a diff in pull request #14099: KAFKA-14702: Extend server side assignor to support rack aware replica placement

2023-07-27 Thread via GitHub
dajac commented on code in PR #14099: URL: https://github.com/apache/kafka/pull/14099#discussion_r1276659754 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -5692,12 +5725,60 @@ private void

[GitHub] [kafka] dajac commented on a diff in pull request #14099: KAFKA-14702: Extend server side assignor to support rack aware replica placement

2023-07-27 Thread via GitHub
dajac commented on code in PR #14099: URL: https://github.com/apache/kafka/pull/14099#discussion_r1276659754 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -5692,12 +5725,60 @@ private void

[GitHub] [kafka] dajac commented on a diff in pull request #14099: KAFKA-14702: Extend server side assignor to support rack aware replica placement

2023-07-27 Thread via GitHub
dajac commented on code in PR #14099: URL: https://github.com/apache/kafka/pull/14099#discussion_r1276653838 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -5567,7 +5600,7 @@ public void

[jira] [Commented] (KAFKA-14748) Relax non-null FK left-join requirement

2023-07-27 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17748285#comment-17748285 ] Guozhang Wang commented on KAFKA-14748: --- [~aki] Thanks for picking this series! I think we can

[jira] [Commented] (KAFKA-12317) Relax non-null key requirement for left/outer KStream joins

2023-07-27 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17748284#comment-17748284 ] Guozhang Wang commented on KAFKA-12317: --- [~mjsax] Though it may not introduce any new configs or

[jira] [Commented] (KAFKA-15240) BrokerToControllerChannelManager cache activeController error cause DefaultAlterPartitionManager send AlterPartition request failed

2023-07-27 Thread Guozhang Wang (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17748281#comment-17748281 ] Guozhang Wang commented on KAFKA-15240: --- [~lushilin] Thanks for reporting this. I think [~cmccabe]

[GitHub] [kafka] ahuang98 commented on a diff in pull request #14084: [MINOR] Add latest versions to kraft upgrade kafkatest

2023-07-27 Thread via GitHub
ahuang98 commented on code in PR #14084: URL: https://github.com/apache/kafka/pull/14084#discussion_r1276628397 ## tests/kafkatest/version.py: ## @@ -252,3 +252,7 @@ def get_version(node=None): V_3_5_0 = KafkaVersion("3.5.0") V_3_5_1 = KafkaVersion("3.5.1") LATEST_3_5 =

[GitHub] [kafka] mumrah commented on a diff in pull request #14115: KAFKA-15263 Check KRaftMigrationDriver state in each event

2023-07-27 Thread via GitHub
mumrah commented on code in PR #14115: URL: https://github.com/apache/kafka/pull/14115#discussion_r1276619004 ## metadata/src/main/java/org/apache/kafka/metadata/migration/KRaftMigrationDriver.java: ## @@ -297,6 +297,16 @@ private boolean isValidStateChange(MigrationDriverState

[GitHub] [kafka] cmccabe commented on a diff in pull request #14115: KAFKA-15263 Check KRaftMigrationDriver state in each event

2023-07-27 Thread via GitHub
cmccabe commented on code in PR #14115: URL: https://github.com/apache/kafka/pull/14115#discussion_r1276615552 ## metadata/src/main/java/org/apache/kafka/metadata/migration/KRaftMigrationDriver.java: ## @@ -297,6 +297,16 @@ private boolean

[GitHub] [kafka] mumrah commented on a diff in pull request #14115: KAFKA-15263 Check KRaftMigrationDriver state in each event

2023-07-27 Thread via GitHub
mumrah commented on code in PR #14115: URL: https://github.com/apache/kafka/pull/14115#discussion_r1276614670 ## metadata/src/main/java/org/apache/kafka/metadata/migration/KRaftMigrationDriver.java: ## @@ -599,6 +604,9 @@ public void run() throws Exception { class

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #14099: KAFKA-14702: Extend server side assignor to support rack aware replica placement

2023-07-27 Thread via GitHub
rreddy-22 commented on code in PR #14099: URL: https://github.com/apache/kafka/pull/14099#discussion_r1276612481 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/RecordHelpersTest.java: ## @@ -612,15 +616,14 @@ public void

[GitHub] [kafka] rondagostino commented on a diff in pull request #14084: [MINOR] Add latest versions to kraft upgrade kafkatest

2023-07-27 Thread via GitHub
rondagostino commented on code in PR #14084: URL: https://github.com/apache/kafka/pull/14084#discussion_r1276607523 ## tests/kafkatest/version.py: ## @@ -252,3 +252,7 @@ def get_version(node=None): V_3_5_0 = KafkaVersion("3.5.0") V_3_5_1 = KafkaVersion("3.5.1") LATEST_3_5 =

[GitHub] [kafka] mumrah opened a new pull request, #14115: KAFKA-15263 Check KRaftMigrationDriver state in each event

2023-07-27 Thread via GitHub
mumrah opened a new pull request, #14115: URL: https://github.com/apache/kafka/pull/14115 To avoid processing events once we have transitioned to a new state, add a check at the beginning of the events created by PollEvent in KRaftMigrationDriver. -- This is an automated message from

[GitHub] [kafka] kamalcph commented on pull request #14114: KAFKA-12969: Add broker level config synonyms for topic level tiered storage configs

2023-07-27 Thread via GitHub
kamalcph commented on PR #14114: URL: https://github.com/apache/kafka/pull/14114#issuecomment-1654008152 @satishd @divijvaidya @showuon PTAL when you get chance! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[jira] [Created] (KAFKA-15263) KRaftMigrationDriver can run the migration twice

2023-07-27 Thread David Arthur (Jira)
David Arthur created KAFKA-15263: Summary: KRaftMigrationDriver can run the migration twice Key: KAFKA-15263 URL: https://issues.apache.org/jira/browse/KAFKA-15263 Project: Kafka Issue Type:

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #14099: KAFKA-14702: Extend server side assignor to support rack aware replica placement

2023-07-27 Thread via GitHub
rreddy-22 commented on code in PR #14099: URL: https://github.com/apache/kafka/pull/14099#discussion_r1276573086 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/consumer/SubscribedTopicMetadata.java: ## @@ -0,0 +1,88 @@ +/* + * Licensed to the Apache

[GitHub] [kafka] kamalcph opened a new pull request, #14114: KAFKA-12969: Add broker level config synonyms for topic level tiered storage configs

2023-07-27 Thread via GitHub
kamalcph opened a new pull request, #14114: URL: https://github.com/apache/kafka/pull/14114 **Topic -> Broker Synonym** - local.retention.bytes -> log.local.retention.bytes - local.retention.ms -> log.local.retention.ms We cannot add synonym for `remote.storage.enable`

[GitHub] [kafka] jolshan merged pull request #14088: MINOR: Adjust Invalid Record Exception for Invalid Txn State as mentioned in KIP-890

2023-07-27 Thread via GitHub
jolshan merged PR #14088: URL: https://github.com/apache/kafka/pull/14088 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] clolov commented on a diff in pull request #14049: KAFKA-14038: Optimise calculation of size for log in remote tier

2023-07-27 Thread via GitHub
clolov commented on code in PR #14049: URL: https://github.com/apache/kafka/pull/14049#discussion_r1276543815 ## storage/api/src/test/java/org/apache/kafka/server/log/remote/storage/NoOpRemoteLogMetadataManager.java: ## @@ -74,6 +74,11 @@ public void

[GitHub] [kafka] philipnee commented on pull request #13920: KAFKA-15106 fix AbstractStickyAssignor isBalanced predict

2023-07-27 Thread via GitHub
philipnee commented on PR #13920: URL: https://github.com/apache/kafka/pull/13920#issuecomment-1653959801 Hmm - you can try. But I've had similar, if not the same, failures and I haven't had a chance to examine the cause. -- This is an automated message from the Apache Git Service. To

[GitHub] [kafka] abhijeetk88 opened a new pull request, #14113: KAFKA-15260: RLM Task should wait for RLMM to initialize

2023-07-27 Thread via GitHub
abhijeetk88 opened a new pull request, #14113: URL: https://github.com/apache/kafka/pull/14113 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of

[jira] [Updated] (KAFKA-15262) MirrorHeartbeatConnector is not working as documented

2023-07-27 Thread Ravindranath Kakarla (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindranath Kakarla updated KAFKA-15262: - Description: As per the MM2

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #14099: KAFKA-14702: Extend server side assignor to support rack aware replica placement

2023-07-27 Thread via GitHub
rreddy-22 commented on code in PR #14099: URL: https://github.com/apache/kafka/pull/14099#discussion_r1276487843 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/consumer/TargetAssignmentBuilderTest.java: ## @@ -624,8 +745,27 @@ public void

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #14099: KAFKA-14702: Extend server side assignor to support rack aware replica placement

2023-07-27 Thread via GitHub
rreddy-22 commented on code in PR #14099: URL: https://github.com/apache/kafka/pull/14099#discussion_r1275444245 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/consumer/TargetAssignmentBuilderTest.java: ## @@ -624,8 +745,27 @@ public void

[jira] [Updated] (KAFKA-15262) MirrorHeartbeatConnector is not working as documented

2023-07-27 Thread Ravindranath Kakarla (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindranath Kakarla updated KAFKA-15262: - Description: As per the MM2

[jira] [Updated] (KAFKA-15262) MirrorHeartbeatConnector is not working as documented

2023-07-27 Thread Ravindranath Kakarla (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindranath Kakarla updated KAFKA-15262: - Description: As per the MM2

[jira] [Updated] (KAFKA-15262) MirrorHeartbeatConnector is not working as documented

2023-07-27 Thread Ravindranath Kakarla (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindranath Kakarla updated KAFKA-15262: - Description: As per the MM2 [KIP-382 |

[jira] [Updated] (KAFKA-15262) MirrorHeartbeatConnector is not working as documented

2023-07-27 Thread Ravindranath Kakarla (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravindranath Kakarla updated KAFKA-15262: - Description: As per the MM2

[jira] [Created] (KAFKA-15262) MirrorHeartbeatConnector is not working as documented

2023-07-27 Thread Ravindranath Kakarla (Jira)
Ravindranath Kakarla created KAFKA-15262: Summary: MirrorHeartbeatConnector is not working as documented Key: KAFKA-15262 URL: https://issues.apache.org/jira/browse/KAFKA-15262 Project: Kafka

[jira] [Comment Edited] (KAFKA-14509) Add ConsumerGroupDescribe API

2023-07-27 Thread Max Riedel (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17748216#comment-17748216 ] Max Riedel edited comment on KAFKA-14509 at 7/27/23 3:40 PM: - Hey [~dajac],

[jira] [Comment Edited] (KAFKA-14509) Add ConsumerGroupDescribe API

2023-07-27 Thread Max Riedel (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17748216#comment-17748216 ] Max Riedel edited comment on KAFKA-14509 at 7/27/23 3:39 PM: - Hey [~dajac],

[jira] [Commented] (KAFKA-14509) Add ConsumerGroupDescribe API

2023-07-27 Thread Max Riedel (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17748216#comment-17748216 ] Max Riedel commented on KAFKA-14509: Hey [~dajac], I started implementing the request/response

[GitHub] [kafka] mimaison commented on pull request #13671: KAFKA-14967: fix NPE in MockAdminClient CreateTopicsResult

2023-07-27 Thread via GitHub
mimaison commented on PR #13671: URL: https://github.com/apache/kafka/pull/13671#issuecomment-1653859939 Thanks @hertzsprung for the PR! No code in Kafka relies on the content of CreateTopicsResult, and I just realized MockAdminClient is not part of the public API so this is effectively

[GitHub] [kafka] abhijeetk88 opened a new pull request, #14112: KAFKA-15261: Do not block replica fetcher if RLMM is not initialized

2023-07-27 Thread via GitHub
abhijeetk88 opened a new pull request, #14112: URL: https://github.com/apache/kafka/pull/14112 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of

[GitHub] [kafka] ex172000 commented on pull request #14110: MINOR: Add test for describe topic with ID

2023-07-27 Thread via GitHub
ex172000 commented on PR #14110: URL: https://github.com/apache/kafka/pull/14110#issuecomment-1653814175 The test failed, but not related. All failed tests are integration test and this is a unit test only change. -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [kafka] jsancio commented on a diff in pull request #13643: KAFKA-15213: provide the exact offset to QuorumController.replay

2023-07-27 Thread via GitHub
jsancio commented on code in PR #13643: URL: https://github.com/apache/kafka/pull/13643#discussion_r1276417011 ## raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java: ## @@ -2394,6 +2394,11 @@ public Optional latestSnapshotId() { return

  1   2   >