[GitHub] [kafka] jeqo commented on a diff in pull request #14127: KAFKA-15181: Wait for RemoteLogMetadataCache to initialize after assigning partitions

jeqo commented on code in PR #14127: URL: https://github.com/apache/kafka/pull/14127#discussion_r1296742080 ## storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/ConsumerTask.java: ## @@ -64,302 +64,395 @@ class ConsumerTask implements Runnable, Closeable

[GitHub] [kafka] satishd commented on a diff in pull request #13561: KAFKA-14888: Added remote log segments retention functionality based on time and size.

satishd commented on code in PR #13561: URL: https://github.com/apache/kafka/pull/13561#discussion_r1296732476 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -696,11 +704,327 @@ public void run() { } } +public void handleLogSt

[GitHub] [kafka] satishd commented on a diff in pull request #13561: KAFKA-14888: Added remote log segments retention functionality based on time and size.

satishd commented on code in PR #13561: URL: https://github.com/apache/kafka/pull/13561#discussion_r1296731885 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -696,11 +704,327 @@ public void run() { } } +public void handleLogSt

[GitHub] [kafka] kamalcph commented on a diff in pull request #14127: KAFKA-15181: Wait for RemoteLogMetadataCache to initialize after assigning partitions

kamalcph commented on code in PR #14127: URL: https://github.com/apache/kafka/pull/14127#discussion_r1296722480 ## storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/ConsumerTask.java: ## @@ -64,302 +64,395 @@ class ConsumerTask implements Runnable, Close

[GitHub] [kafka] jeqo commented on a diff in pull request #14127: KAFKA-15181: Wait for RemoteLogMetadataCache to initialize after assigning partitions

jeqo commented on code in PR #14127: URL: https://github.com/apache/kafka/pull/14127#discussion_r1296717997 ## storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/ConsumerTask.java: ## @@ -64,302 +64,395 @@ class ConsumerTask implements Runnable, Closeable

[GitHub] [kafka] jeqo commented on a diff in pull request #14127: KAFKA-15181: Wait for RemoteLogMetadataCache to initialize after assigning partitions

jeqo commented on code in PR #14127: URL: https://github.com/apache/kafka/pull/14127#discussion_r1296717997 ## storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/ConsumerTask.java: ## @@ -64,302 +64,395 @@ class ConsumerTask implements Runnable, Closeable

[GitHub] [kafka] kamalcph commented on a diff in pull request #13947: KAFKA-15130: Delete remote segments when delete a topic

kamalcph commented on code in PR #13947: URL: https://github.com/apache/kafka/pull/13947#discussion_r1296367515 ## core/src/test/scala/integration/kafka/admin/RemoteTopicDeletionTest.scala: ## @@ -0,0 +1,148 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one o

[GitHub] [kafka] kamalcph commented on a diff in pull request #14127: KAFKA-15181: Wait for RemoteLogMetadataCache to initialize after assigning partitions

kamalcph commented on code in PR #14127: URL: https://github.com/apache/kafka/pull/14127#discussion_r1296661660 ## storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/ConsumerTask.java: ## @@ -64,302 +64,395 @@ class ConsumerTask implements Runnable, Close

[GitHub] [kafka] kamalcph commented on a diff in pull request #14127: KAFKA-15181: Wait for RemoteLogMetadataCache to initialize after assigning partitions

kamalcph commented on code in PR #14127: URL: https://github.com/apache/kafka/pull/14127#discussion_r1296661660 ## storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/ConsumerTask.java: ## @@ -64,302 +64,395 @@ class ConsumerTask implements Runnable, Close

[GitHub] [kafka] kamalcph commented on a diff in pull request #14161: KAFKA-15267: Do not allow Tiered Storage to be disabled while topics have remote.storage.enable property

kamalcph commented on code in PR #14161: URL: https://github.com/apache/kafka/pull/14161#discussion_r1296140643 ## core/src/main/scala/kafka/server/ConfigHandler.scala: ## @@ -62,6 +62,15 @@ class TopicConfigHandler(private val logManager: LogManager, kafkaConfig: KafkaC t

[jira] [Comment Edited] (KAFKA-15046) Produce performance issue under high disk load

[ https://issues.apache.org/jira/browse/KAFKA-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17755351#comment-17755351 ] Haruki Okada edited comment on KAFKA-15046 at 8/17/23 4:09 AM: ---

[jira] [Comment Edited] (KAFKA-15046) Produce performance issue under high disk load

[ https://issues.apache.org/jira/browse/KAFKA-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17755351#comment-17755351 ] Haruki Okada edited comment on KAFKA-15046 at 8/17/23 4:08 AM: ---

[GitHub] [kafka] kamalcph commented on pull request #14116: KAFKA-15167: Tiered Storage Test Harness Framework

kamalcph commented on PR #14116: URL: https://github.com/apache/kafka/pull/14116#issuecomment-1681581185 @satishd Can you please take a look when you get chance? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[jira] [Commented] (KAFKA-15046) Produce performance issue under high disk load

[ https://issues.apache.org/jira/browse/KAFKA-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17755351#comment-17755351 ] Haruki Okada commented on KAFKA-15046: -- [~junrao] Hi, sorry for the late response.

[GitHub] [kafka] showuon commented on pull request #14232: MINOR: Use "add-exports" only when jdk >= 16 in minikdc

showuon commented on PR #14232: URL: https://github.com/apache/kafka/pull/14232#issuecomment-1681580160 Verified in jdk 8/11/17 env. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[GitHub] [kafka] showuon commented on pull request #14011: MINOR: Fix MiniKdc Java 17 issue in system tests

showuon commented on PR #14011: URL: https://github.com/apache/kafka/pull/14011#issuecomment-1681579990 Good call @gharris1727 ! Created https://github.com/apache/kafka/pull/14232 to fix it. -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [kafka] showuon opened a new pull request, #14232: MINOR: Use "add-exports" only when jdk >= 16 in minikdc

showuon opened a new pull request, #14232: URL: https://github.com/apache/kafka/pull/14232 Use "add-exports" only when jdk >= 16 in minikdc ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build statu

[GitHub] [kafka] github-actions[bot] commented on pull request #13682: MINOR: improved exception/warn logging for stream-stream join store settings

github-actions[bot] commented on PR #13682: URL: https://github.com/apache/kafka/pull/13682#issuecomment-1681558951 This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please ask a committer for review. If the PR has merge

[jira] [Updated] (KAFKA-15353) Empty ISR returned from controller after AlterPartition request

[ https://issues.apache.org/jira/browse/KAFKA-15353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen updated KAFKA-15353: -- Description: In [KIP-903|https://cwiki.apache.org/confluence/display/KAFKA/KIP-903%3A+Replicas+with+s

[jira] [Updated] (KAFKA-15353) Empty ISR returned from controller after AlterPartition request

[ https://issues.apache.org/jira/browse/KAFKA-15353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen updated KAFKA-15353: -- Description: In [KIP-903|https://cwiki.apache.org/confluence/display/KAFKA/KIP-903%3A+Replicas+with+s

[jira] [Updated] (KAFKA-15353) Empty ISR returned from controller after AlterPartition request

[ https://issues.apache.org/jira/browse/KAFKA-15353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen updated KAFKA-15353: -- Description: In [KIP-903|https://cwiki.apache.org/confluence/display/KAFKA/KIP-903%3A+Replicas+with+s

[jira] [Created] (KAFKA-15371) MetadataShell is stuck when bootstrapping

Deng Ziming created KAFKA-15371: --- Summary: MetadataShell is stuck when bootstrapping Key: KAFKA-15371 URL: https://issues.apache.org/jira/browse/KAFKA-15371 Project: Kafka Issue Type: Bug A

[GitHub] [kafka] Owen-CH-Leung commented on pull request #14136: Add metadatacache into RemoteLogManager, and refactor all relevant codes

Owen-CH-Leung commented on PR #14136: URL: https://github.com/apache/kafka/pull/14136#issuecomment-1681516264 > Overall LGTM. Can we hold this patch until #13947 lands? Otherwise, this patch needs one integration test to verify end-to-end change. No problem. I can perform rebase after

[GitHub] [kafka] Owen-CH-Leung commented on a diff in pull request #14136: Add metadatacache into RemoteLogManager, and refactor all relevant codes

Owen-CH-Leung commented on code in PR #14136: URL: https://github.com/apache/kafka/pull/14136#discussion_r1296606986 ## core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala: ## @@ -3600,7 +3600,10 @@ class ReplicaManagerTest { "clusterId", time, _

[GitHub] [kafka] Owen-CH-Leung commented on a diff in pull request #14136: Add metadatacache into RemoteLogManager, and refactor all relevant codes

Owen-CH-Leung commented on code in PR #14136: URL: https://github.com/apache/kafka/pull/14136#discussion_r1296606546 ## core/src/test/java/kafka/log/remote/RemoteLogManagerTest.java: ## @@ -802,41 +817,25 @@ private void verifyInCache(TopicIdPartition... topicIdPartitions) {

[GitHub] [kafka] showuon commented on pull request #14116: KAFKA-15167: Tiered Storage Test Harness Framework

showuon commented on PR #14116: URL: https://github.com/apache/kafka/pull/14116#issuecomment-1681476462 @kamalcph , sorry, I'll attend a conference the following days, so I can't review this until next Tuesday. Does this PR block the following PRs? If so, I think since Divij already approve

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1296578432 ## metadata/src/main/java/org/apache/kafka/controller/DelegationTokenControlManager.java: ## @@ -0,0 +1,359 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1296577535 ## metadata/src/main/java/org/apache/kafka/controller/DelegationTokenControlManager.java: ## @@ -0,0 +1,359 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1296577263 ## metadata/src/main/java/org/apache/kafka/controller/DelegationTokenControlManager.java: ## @@ -0,0 +1,359 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1296576872 ## metadata/src/main/java/org/apache/kafka/controller/DelegationTokenControlManager.java: ## @@ -0,0 +1,359 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1296576768 ## metadata/src/main/java/org/apache/kafka/controller/DelegationTokenControlManager.java: ## @@ -0,0 +1,359 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1296576660 ## metadata/src/main/java/org/apache/kafka/controller/DelegationTokenControlManager.java: ## @@ -0,0 +1,359 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1296576394 ## metadata/src/main/java/org/apache/kafka/controller/QuorumController.java: ## @@ -202,6 +211,11 @@ static public class Builder { private BootstrapMetadat

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1296572585 ## metadata/src/test/java/org/apache/kafka/image/DelegationTokenImageTest.java: ## @@ -0,0 +1,141 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] [kafka] hachikuji commented on a diff in pull request #14213: KAFKA-15345; KRaft leader notifies when listener reaches epoch start

hachikuji commented on code in PR #14213: URL: https://github.com/apache/kafka/pull/14213#discussion_r1296563672 ## raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java: ## @@ -2653,11 +2652,11 @@ private boolean shouldFireLeaderChange(LeaderAndEpoch leaderAndEpoch) {

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1296567527 ## metadata/src/main/java/org/apache/kafka/image/node/DelegationTokenDataNode.java: ## Review Comment: There are unit tests to check the redaction state. See

[jira] [Created] (KAFKA-15370) Support Participation in 2PC (KIP-939)

Artem Livshits created KAFKA-15370: -- Summary: Support Participation in 2PC (KIP-939) Key: KAFKA-15370 URL: https://issues.apache.org/jira/browse/KAFKA-15370 Project: Kafka Issue Type: Improv

[GitHub] [kafka] artemlivshits opened a new pull request, #14231: [DRAFT] EXAMPLE: dual write recipe example for KIP-939

artemlivshits opened a new pull request, #14231: URL: https://github.com/apache/kafka/pull/14231 This example uses JDBC and KafkaProducer to implement atomic dual writes to a SQL database and Kafka. The code assumes that KIP-890 part 2 is also implemented, so Kafka commit would bump the ep

[GitHub] [kafka] gharris1727 opened a new pull request, #14230: KAFKA-15226: Add connect-plugin-path and plugin.discovery system test

gharris1727 opened a new pull request, #14230: URL: https://github.com/apache/kafka/pull/14230 This test uses a standalone worker to verify the behavior of the plugin.discovery configuration values with non-migrated and migrated artifacts. For a source of non-migrated artifacts, I'm

[jira] [Commented] (KAFKA-15302) Stale value returned when using store.all() in punctuation function.

[ https://issues.apache.org/jira/browse/KAFKA-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17755298#comment-17755298 ] Matthias J. Sax commented on KAFKA-15302: - I would need to think more about this

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1296512550 ## metadata/src/main/java/org/apache/kafka/controller/OffsetControlManager.java: ## @@ -212,10 +213,14 @@ long lastStableOffset() { } /** - * @return th

[GitHub] [kafka] tanay27 opened a new pull request, #14229: KAFKA-15918: automating version change

tanay27 opened a new pull request, #14229: URL: https://github.com/apache/kafka/pull/14229 Created a new folder called `release` and moved `release.py`, `release_notes.py` and `reviewers.py` to the new folder. Created two new files: - `requirements.txt` to support all the requi

[jira] [Resolved] (KAFKA-15220) KRaftMetadataCache returns fenced brokers from getAliveBrokerNode

[ https://issues.apache.org/jira/browse/KAFKA-15220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-15220. -- Fix Version/s: 3.6.0 Resolution: Fixed > KRaftMetadataCache returns fenced brokers from

[jira] [Created] (KAFKA-15369) Allow AdminClient to Talk Directly with the KRaft Controller Quorum and add Controller Registration

Colin McCabe created KAFKA-15369: Summary: Allow AdminClient to Talk Directly with the KRaft Controller Quorum and add Controller Registration Key: KAFKA-15369 URL: https://issues.apache.org/jira/browse/KAFKA-1536

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1296410110 ## metadata/src/main/java/org/apache/kafka/controller/DelegationTokenControlManager.java: ## @@ -0,0 +1,359 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1296409238 ## metadata/src/main/java/org/apache/kafka/controller/DelegationTokenControlManager.java: ## @@ -0,0 +1,359 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1296399634 ## metadata/src/main/java/org/apache/kafka/controller/DelegationTokenControlManager.java: ## @@ -0,0 +1,359 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1296382585 ## metadata/src/main/java/org/apache/kafka/controller/DelegationTokenControlManager.java: ## @@ -0,0 +1,359 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1296379603 ## metadata/src/main/java/org/apache/kafka/controller/DelegationTokenControlManager.java: ## @@ -0,0 +1,359 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1296376127 ## metadata/src/main/java/org/apache/kafka/controller/DelegationTokenControlManager.java: ## @@ -0,0 +1,359 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] [kafka] jeqo commented on a diff in pull request #12637: KAFKA-14226 (KIP-821): [connect:transform] Introduce support for nested structures

jeqo commented on code in PR #12637: URL: https://github.com/apache/kafka/pull/12637#discussion_r1296374731 ## connect/transforms/src/main/java/org/apache/kafka/connect/transforms/field/FieldPath.java: ## @@ -0,0 +1,232 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [kafka] kamalcph commented on pull request #14116: KAFKA-15167: Tiered Storage Test Harness Framework

kamalcph commented on PR #14116: URL: https://github.com/apache/kafka/pull/14116#issuecomment-1681198112 @showuon Call for review. PTAL. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[GitHub] [kafka] kamalcph commented on pull request #13947: KAFKA-15130: Delete remote segments when delete a topic

kamalcph commented on PR #13947: URL: https://github.com/apache/kafka/pull/13947#issuecomment-1681195520 @divijvaidya @junrao @showuon @satishd Addressed your review comments. PTAL. -- This is an automated message from the Apache Git Service. To respond to the message, please log o

[GitHub] [kafka] kamalcph commented on a diff in pull request #13947: KAFKA-15130: Delete remote segments when delete a topic

kamalcph commented on code in PR #13947: URL: https://github.com/apache/kafka/pull/13947#discussion_r1296367515 ## core/src/test/scala/integration/kafka/admin/RemoteTopicDeletionTest.scala: ## @@ -0,0 +1,148 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one o

[GitHub] [kafka] kamalcph commented on a diff in pull request #13947: KAFKA-15130: Delete remote segments when delete a topic

kamalcph commented on code in PR #13947: URL: https://github.com/apache/kafka/pull/13947#discussion_r1296367515 ## core/src/test/scala/integration/kafka/admin/RemoteTopicDeletionTest.scala: ## @@ -0,0 +1,148 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one o

[GitHub] [kafka] kamalcph commented on a diff in pull request #13947: KAFKA-15130: Delete remote segments when delete a topic

kamalcph commented on code in PR #13947: URL: https://github.com/apache/kafka/pull/13947#discussion_r1296359728 ## storage/src/main/java/org/apache/kafka/storage/internals/log/RemoteIndexCache.java: ## @@ -166,6 +166,10 @@ public Cache internalCache() { return internalC

[GitHub] [kafka] kamalcph commented on a diff in pull request #13947: KAFKA-15130: Delete remote segments when delete a topic

kamalcph commented on code in PR #13947: URL: https://github.com/apache/kafka/pull/13947#discussion_r1296357957 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -343,21 +345,80 @@ public void onLeadershipChange(Set partitionsBecomeLeader, /** * Del

[GitHub] [kafka] kamalcph commented on a diff in pull request #13947: KAFKA-15130: Delete remote segments when delete a topic

kamalcph commented on code in PR #13947: URL: https://github.com/apache/kafka/pull/13947#discussion_r1296356436 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -343,21 +345,80 @@ public void onLeadershipChange(Set partitionsBecomeLeader, /** * Del

[GitHub] [kafka] kamalcph commented on a diff in pull request #13947: KAFKA-15130: Delete remote segments when delete a topic

kamalcph commented on code in PR #13947: URL: https://github.com/apache/kafka/pull/13947#discussion_r1296356436 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -343,21 +345,80 @@ public void onLeadershipChange(Set partitionsBecomeLeader, /** * Del

[GitHub] [kafka] kamalcph commented on a diff in pull request #13947: KAFKA-15130: Delete remote segments when delete a topic

kamalcph commented on code in PR #13947: URL: https://github.com/apache/kafka/pull/13947#discussion_r1296353494 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -343,21 +345,80 @@ public void onLeadershipChange(Set partitionsBecomeLeader, /** * Del

[GitHub] [kafka] kamalcph commented on a diff in pull request #13947: KAFKA-15130: Delete remote segments when delete a topic

kamalcph commented on code in PR #13947: URL: https://github.com/apache/kafka/pull/13947#discussion_r1296354652 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -343,21 +345,80 @@ public void onLeadershipChange(Set partitionsBecomeLeader, /** * Del

[GitHub] [kafka] kamalcph commented on a diff in pull request #13947: KAFKA-15130: Delete remote segments when delete a topic

kamalcph commented on code in PR #13947: URL: https://github.com/apache/kafka/pull/13947#discussion_r1296353850 ## storage/src/main/java/org/apache/kafka/storage/internals/log/RemoteIndexCache.java: ## @@ -421,6 +425,7 @@ public void close() { // Initiate shutdo

[GitHub] [kafka] kamalcph commented on a diff in pull request #13947: KAFKA-15130: Delete remote segments when delete a topic

kamalcph commented on code in PR #13947: URL: https://github.com/apache/kafka/pull/13947#discussion_r1296353494 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -343,21 +345,80 @@ public void onLeadershipChange(Set partitionsBecomeLeader, /** * Del

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1296352968 ## metadata/src/main/java/org/apache/kafka/controller/DelegationTokenControlManager.java: ## @@ -0,0 +1,359 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1296349986 ## metadata/src/main/java/org/apache/kafka/controller/DelegationTokenControlManager.java: ## @@ -0,0 +1,359 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] [kafka] hgeraldino commented on pull request #14093: KAFKA-15248 Add BooleanConverter

hgeraldino commented on PR #14093: URL: https://github.com/apache/kafka/pull/14093#issuecomment-1681170703 > Thanks @hgeraldino, LGTM! I just had a few more minor suggestions. Addressed the second round of comments. Thanks again! -- This is an automated message from the Apache Git S

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #14182: KAFKA 14515: Optimized Uniform Rack Aware Assignor

jeffkbkim commented on code in PR #14182: URL: https://github.com/apache/kafka/pull/14182#discussion_r1296343128 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/OptimizedUniformAssignmentBuilder.java: ## @@ -0,0 +1,399 @@ +/* + * Licensed to the Ap

[GitHub] [kafka] rondagostino commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

rondagostino commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1296233969 ## metadata/src/main/java/org/apache/kafka/metadata/DelegationTokenData.java: ## Review Comment: Is a Unit test necessary, do you think? I also realize we do

[jira] [Resolved] (KAFKA-15228) Add sync-manifests subcommand to connect-plugin-path tool

[ https://issues.apache.org/jira/browse/KAFKA-15228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Harris resolved KAFKA-15228. - Resolution: Fixed > Add sync-manifests subcommand to connect-plugin-path tool >

[GitHub] [kafka] gharris1727 merged pull request #14195: KAFKA-15228: Add sync-manifests command to connect-plugin-path (KIP-898)

gharris1727 merged PR #14195: URL: https://github.com/apache/kafka/pull/14195 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apa

[GitHub] [kafka] bachmanity1 commented on pull request #14228: MINOR: Fix ReplicationBytes(Out|In)PerSec metrics' docs

bachmanity1 commented on PR #14228: URL: https://github.com/apache/kafka/pull/14228#issuecomment-1681087535 @mimaison @ijuma @dongjinleekr can you please review this? Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [kafka] bachmanity1 opened a new pull request, #14228: MINOR: Fix ReplicationBytes(Out|In)PerSec metrics' docs

bachmanity1 opened a new pull request, #14228: URL: https://github.com/apache/kafka/pull/14228 Based on the code shown below it looks like `ReplicationBytes(Out|In)PerSec` metrics are reported only across all topics and NOT for each individual topic. https://github.com/apache/kafka/blob/

[GitHub] [kafka] ijuma commented on pull request #14011: MINOR: Fix MiniKdc Java 17 issue in system tests

ijuma commented on PR #14011: URL: https://github.com/apache/kafka/pull/14011#issuecomment-1681073825 @gharris1727 you're right, this option should only be provided if the jdk is 9 or newer. My bad for missing that. -- This is an automated message from the Apache Git Service. To respond t

[GitHub] [kafka] divijvaidya commented on a diff in pull request #13561: KAFKA-14888: Added remote log segments retention functionality based on time and size.

divijvaidya commented on code in PR #13561: URL: https://github.com/apache/kafka/pull/13561#discussion_r1296264710 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -696,11 +704,327 @@ public void run() { } } +public void handleL

[GitHub] [kafka] divijvaidya commented on a diff in pull request #13561: KAFKA-14888: Added remote log segments retention functionality based on time and size.

divijvaidya commented on code in PR #13561: URL: https://github.com/apache/kafka/pull/13561#discussion_r1296264710 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -696,11 +704,327 @@ public void run() { } } +public void handleL

[jira] [Commented] (KAFKA-15223) Need more clarity in documentation for upgrade/downgrade procedures and limitations across releases.

[ https://issues.apache.org/jira/browse/KAFKA-15223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17755207#comment-17755207 ] kaushik srinivas commented on KAFKA-15223: -- Any updates on this ? > Need more

[GitHub] [kafka] cadonna commented on a diff in pull request #13927: [DO NOT MERGE] KAFKA-10199: Enable state updater by default

cadonna commented on code in PR #13927: URL: https://github.com/apache/kafka/pull/13927#discussion_r1296232439 ## streams/src/test/java/org/apache/kafka/streams/integration/PurgeRepartitionTopicIntegrationTest.java: ## @@ -212,10 +213,11 @@ public void shouldRestoreState() throw

[GitHub] [kafka] mumrah commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

mumrah commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1296231483 ## metadata/src/main/java/org/apache/kafka/controller/OffsetControlManager.java: ## @@ -212,10 +213,14 @@ long lastStableOffset() { } /** - * @return the

[GitHub] [kafka] gharris1727 merged pull request #13302: KAFKA-14759: Move Mock, Schema, and Verifiable connectors to new test-plugins module

gharris1727 merged PR #13302: URL: https://github.com/apache/kafka/pull/13302 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apa

[GitHub] [kafka] gharris1727 commented on pull request #13302: KAFKA-14759: Move Mock, Schema, and Verifiable connectors to new test-plugins module

gharris1727 commented on PR #13302: URL: https://github.com/apache/kafka/pull/13302#issuecomment-1681010261 Test failures in CI appear unrelated. I ran the Connect system tests for this change with #14011 reverted and they all passed. -- This is an automated message from the Apache Git Se

[GitHub] [kafka] gharris1727 commented on pull request #14220: KAFKA-15102: Add release notes about the replication.policy.internal.topic.separator.enabled property for MirrorMaker 2

gharris1727 commented on PR #14220: URL: https://github.com/apache/kafka/pull/14220#issuecomment-1681003739 > I'd just cherry-pick the commit for this PR, but we don't have autogenerated docs for the MM2 properties on those branches, so the wording will have to be different. Yes the

[GitHub] [kafka] junrao commented on a diff in pull request #13561: KAFKA-14888: Added remote log segments retention functionality based on time and size.

junrao commented on code in PR #13561: URL: https://github.com/apache/kafka/pull/13561#discussion_r1296221075 ## core/src/main/scala/kafka/log/UnifiedLog.scala: ## @@ -1390,7 +1424,15 @@ class UnifiedLog(@volatile var logStartOffset: Long, private def deleteOldSegments(predic

[GitHub] [kafka] mumrah commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

mumrah commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1296220658 ## metadata/src/main/java/org/apache/kafka/controller/QuorumController.java: ## @@ -892,48 +915,73 @@ class MigrationWriteOperation implements ControllerWriteOperation

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1296172460 ## metadata/src/main/java/org/apache/kafka/controller/QuorumController.java: ## @@ -892,48 +915,73 @@ class MigrationWriteOperation implements ControllerWriteOperation

[GitHub] [kafka] cadonna opened a new pull request, #14227: MINOR: Decouple purging committed records from committing

cadonna opened a new pull request, #14227: URL: https://github.com/apache/kafka/pull/14227 Currently, Kafka Streams only tries to purge records whose offset are committed from a repartition topic when at least one offset was committed in the current commit. The coupling between committing s

[GitHub] [kafka] lucasbru opened a new pull request, #14226: Kafka Streams Threading: Timeout behavior

lucasbru opened a new pull request, #14226: URL: https://github.com/apache/kafka/pull/14226 The process method inside the tasks needs to be called from within the processing threads. However, it currently interacts with the consumer in two ways: - It resumes processing when the

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1296200794 ## metadata/src/main/java/org/apache/kafka/image/loader/LogDeltaManifest.java: ## @@ -66,6 +119,10 @@ public LogDeltaManifest( this.numBytes = numBytes; }

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r129629 ## metadata/src/main/java/org/apache/kafka/controller/OffsetControlManager.java: ## @@ -212,10 +213,14 @@ long lastStableOffset() { } /** - * @return th

[jira] [Commented] (KAFKA-15338) The metric group documentation for metrics added in KAFKA-13945 is incorrect

[ https://issues.apache.org/jira/browse/KAFKA-15338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17755184#comment-17755184 ] Matthias J. Sax commented on KAFKA-15338: - Sure. Thank a lot! > The metric grou

[GitHub] [kafka] gharris1727 commented on a diff in pull request #14220: KAFKA-15102: Add release notes about the replication.policy.internal.topic.separator.enabled property for MirrorMaker 2

gharris1727 commented on code in PR #14220: URL: https://github.com/apache/kafka/pull/14220#discussion_r1296189852 ## docs/upgrade.html: ## @@ -43,6 +43,10 @@ Notable changes in 3 See https://cwiki.apache.org/confluence/display/KAFKA/KIP-925%3A+Rack+aware+task+assig

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1296178891 ## metadata/src/main/java/org/apache/kafka/controller/QuorumController.java: ## @@ -892,48 +915,73 @@ class MigrationWriteOperation implements ControllerWriteOperation

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1296178891 ## metadata/src/main/java/org/apache/kafka/controller/QuorumController.java: ## @@ -892,48 +915,73 @@ class MigrationWriteOperation implements ControllerWriteOperation

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1296178891 ## metadata/src/main/java/org/apache/kafka/controller/QuorumController.java: ## @@ -892,48 +915,73 @@ class MigrationWriteOperation implements ControllerWriteOperation

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1296173796 ## metadata/src/main/java/org/apache/kafka/controller/QuorumController.java: ## @@ -892,48 +915,73 @@ class MigrationWriteOperation implements ControllerWriteOperation

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1296172460 ## metadata/src/main/java/org/apache/kafka/controller/QuorumController.java: ## @@ -892,48 +915,73 @@ class MigrationWriteOperation implements ControllerWriteOperation

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1296173796 ## metadata/src/main/java/org/apache/kafka/controller/QuorumController.java: ## @@ -892,48 +915,73 @@ class MigrationWriteOperation implements ControllerWriteOperation

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1296172460 ## metadata/src/main/java/org/apache/kafka/controller/QuorumController.java: ## @@ -892,48 +915,73 @@ class MigrationWriteOperation implements ControllerWriteOperation

[GitHub] [kafka] kamalcph commented on a diff in pull request #14136: Add metadatacache into RemoteLogManager, and refactor all relevant codes

kamalcph commented on code in PR #14136: URL: https://github.com/apache/kafka/pull/14136#discussion_r1296153252 ## core/src/test/java/kafka/log/remote/RemoteLogManagerTest.java: ## @@ -802,41 +817,25 @@ private void verifyInCache(TopicIdPartition... topicIdPartitions) {

[GitHub] [kafka] nikramakrishnan commented on a diff in pull request #14212: MINOR: Add test case for follower fetch

nikramakrishnan commented on code in PR #14212: URL: https://github.com/apache/kafka/pull/14212#discussion_r1296157690 ## core/src/test/scala/unit/kafka/server/ReplicaFetcherTierStateMachineTest.scala: ## @@ -85,6 +85,73 @@ class ReplicaFetcherTierStateMachineTest { assertE

[GitHub] [kafka] nikramakrishnan commented on a diff in pull request #14212: MINOR: Add test case for follower fetch

nikramakrishnan commented on code in PR #14212: URL: https://github.com/apache/kafka/pull/14212#discussion_r1295970964 ## core/src/test/scala/unit/kafka/server/ReplicaFetcherTierStateMachineTest.scala: ## @@ -85,6 +85,64 @@ class ReplicaFetcherTierStateMachineTest { assertE

  1   2   >