[GitHub] [kafka] kamalcph commented on a diff in pull request #13947: KAFKA-15130: Delete remote segments when delete a topic

2023-08-16 Thread via GitHub
kamalcph commented on code in PR #13947: URL: https://github.com/apache/kafka/pull/13947#discussion_r1296367515 ## core/src/test/scala/integration/kafka/admin/RemoteTopicDeletionTest.scala: ## @@ -0,0 +1,148 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [kafka] kamalcph commented on a diff in pull request #14127: KAFKA-15181: Wait for RemoteLogMetadataCache to initialize after assigning partitions

2023-08-16 Thread via GitHub
kamalcph commented on code in PR #14127: URL: https://github.com/apache/kafka/pull/14127#discussion_r1296661660 ## storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/ConsumerTask.java: ## @@ -64,302 +64,395 @@ class ConsumerTask implements Runnable,

[GitHub] [kafka] kamalcph commented on a diff in pull request #14127: KAFKA-15181: Wait for RemoteLogMetadataCache to initialize after assigning partitions

2023-08-16 Thread via GitHub
kamalcph commented on code in PR #14127: URL: https://github.com/apache/kafka/pull/14127#discussion_r1296661660 ## storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/ConsumerTask.java: ## @@ -64,302 +64,395 @@ class ConsumerTask implements Runnable,

[GitHub] [kafka] kamalcph commented on a diff in pull request #14161: KAFKA-15267: Do not allow Tiered Storage to be disabled while topics have remote.storage.enable property

2023-08-16 Thread via GitHub
kamalcph commented on code in PR #14161: URL: https://github.com/apache/kafka/pull/14161#discussion_r1296140643 ## core/src/main/scala/kafka/server/ConfigHandler.scala: ## @@ -62,6 +62,15 @@ class TopicConfigHandler(private val logManager: LogManager, kafkaConfig: KafkaC

[jira] [Comment Edited] (KAFKA-15046) Produce performance issue under high disk load

2023-08-16 Thread Haruki Okada (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755351#comment-17755351 ] Haruki Okada edited comment on KAFKA-15046 at 8/17/23 4:09 AM: --- [~junrao]

[jira] [Comment Edited] (KAFKA-15046) Produce performance issue under high disk load

2023-08-16 Thread Haruki Okada (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755351#comment-17755351 ] Haruki Okada edited comment on KAFKA-15046 at 8/17/23 4:08 AM: --- [~junrao]

[GitHub] [kafka] kamalcph commented on pull request #14116: KAFKA-15167: Tiered Storage Test Harness Framework

2023-08-16 Thread via GitHub
kamalcph commented on PR #14116: URL: https://github.com/apache/kafka/pull/14116#issuecomment-1681581185 @satishd Can you please take a look when you get chance? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[jira] [Commented] (KAFKA-15046) Produce performance issue under high disk load

2023-08-16 Thread Haruki Okada (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755351#comment-17755351 ] Haruki Okada commented on KAFKA-15046: -- [~junrao] Hi, sorry for the late response. Thanks for your

[GitHub] [kafka] showuon commented on pull request #14232: MINOR: Use "add-exports" only when jdk >= 16 in minikdc

2023-08-16 Thread via GitHub
showuon commented on PR #14232: URL: https://github.com/apache/kafka/pull/14232#issuecomment-1681580160 Verified in jdk 8/11/17 env. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [kafka] showuon commented on pull request #14011: MINOR: Fix MiniKdc Java 17 issue in system tests

2023-08-16 Thread via GitHub
showuon commented on PR #14011: URL: https://github.com/apache/kafka/pull/14011#issuecomment-1681579990 Good call @gharris1727 ! Created https://github.com/apache/kafka/pull/14232 to fix it. -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [kafka] showuon opened a new pull request, #14232: MINOR: Use "add-exports" only when jdk >= 16 in minikdc

2023-08-16 Thread via GitHub
showuon opened a new pull request, #14232: URL: https://github.com/apache/kafka/pull/14232 Use "add-exports" only when jdk >= 16 in minikdc ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build

[GitHub] [kafka] github-actions[bot] commented on pull request #13682: MINOR: improved exception/warn logging for stream-stream join store settings

2023-08-16 Thread via GitHub
github-actions[bot] commented on PR #13682: URL: https://github.com/apache/kafka/pull/13682#issuecomment-1681558951 This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please ask a committer for review. If the PR has

[jira] [Updated] (KAFKA-15353) Empty ISR returned from controller after AlterPartition request

2023-08-16 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen updated KAFKA-15353: -- Description: In

[jira] [Updated] (KAFKA-15353) Empty ISR returned from controller after AlterPartition request

2023-08-16 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen updated KAFKA-15353: -- Description: In

[jira] [Updated] (KAFKA-15353) Empty ISR returned from controller after AlterPartition request

2023-08-16 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen updated KAFKA-15353: -- Description: In

[jira] [Created] (KAFKA-15371) MetadataShell is stuck when bootstrapping

2023-08-16 Thread Deng Ziming (Jira)
Deng Ziming created KAFKA-15371: --- Summary: MetadataShell is stuck when bootstrapping Key: KAFKA-15371 URL: https://issues.apache.org/jira/browse/KAFKA-15371 Project: Kafka Issue Type: Bug

[GitHub] [kafka] Owen-CH-Leung commented on pull request #14136: Add metadatacache into RemoteLogManager, and refactor all relevant codes

2023-08-16 Thread via GitHub
Owen-CH-Leung commented on PR #14136: URL: https://github.com/apache/kafka/pull/14136#issuecomment-1681516264 > Overall LGTM. Can we hold this patch until #13947 lands? Otherwise, this patch needs one integration test to verify end-to-end change. No problem. I can perform rebase

[GitHub] [kafka] Owen-CH-Leung commented on a diff in pull request #14136: Add metadatacache into RemoteLogManager, and refactor all relevant codes

2023-08-16 Thread via GitHub
Owen-CH-Leung commented on code in PR #14136: URL: https://github.com/apache/kafka/pull/14136#discussion_r1296606986 ## core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala: ## @@ -3600,7 +3600,10 @@ class ReplicaManagerTest { "clusterId", time, _

[GitHub] [kafka] Owen-CH-Leung commented on a diff in pull request #14136: Add metadatacache into RemoteLogManager, and refactor all relevant codes

2023-08-16 Thread via GitHub
Owen-CH-Leung commented on code in PR #14136: URL: https://github.com/apache/kafka/pull/14136#discussion_r1296606546 ## core/src/test/java/kafka/log/remote/RemoteLogManagerTest.java: ## @@ -802,41 +817,25 @@ private void verifyInCache(TopicIdPartition... topicIdPartitions) {

[GitHub] [kafka] showuon commented on pull request #14116: KAFKA-15167: Tiered Storage Test Harness Framework

2023-08-16 Thread via GitHub
showuon commented on PR #14116: URL: https://github.com/apache/kafka/pull/14116#issuecomment-1681476462 @kamalcph , sorry, I'll attend a conference the following days, so I can't review this until next Tuesday. Does this PR block the following PRs? If so, I think since Divij already

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-16 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1296578432 ## metadata/src/main/java/org/apache/kafka/controller/DelegationTokenControlManager.java: ## @@ -0,0 +1,359 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-16 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1296577535 ## metadata/src/main/java/org/apache/kafka/controller/DelegationTokenControlManager.java: ## @@ -0,0 +1,359 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-16 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1296577263 ## metadata/src/main/java/org/apache/kafka/controller/DelegationTokenControlManager.java: ## @@ -0,0 +1,359 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-16 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1296576872 ## metadata/src/main/java/org/apache/kafka/controller/DelegationTokenControlManager.java: ## @@ -0,0 +1,359 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-16 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1296576768 ## metadata/src/main/java/org/apache/kafka/controller/DelegationTokenControlManager.java: ## @@ -0,0 +1,359 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-16 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1296576660 ## metadata/src/main/java/org/apache/kafka/controller/DelegationTokenControlManager.java: ## @@ -0,0 +1,359 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-16 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1296576394 ## metadata/src/main/java/org/apache/kafka/controller/QuorumController.java: ## @@ -202,6 +211,11 @@ static public class Builder { private

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-16 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1296572585 ## metadata/src/test/java/org/apache/kafka/image/DelegationTokenImageTest.java: ## @@ -0,0 +1,141 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] [kafka] hachikuji commented on a diff in pull request #14213: KAFKA-15345; KRaft leader notifies when listener reaches epoch start

2023-08-16 Thread via GitHub
hachikuji commented on code in PR #14213: URL: https://github.com/apache/kafka/pull/14213#discussion_r1296563672 ## raft/src/main/java/org/apache/kafka/raft/KafkaRaftClient.java: ## @@ -2653,11 +2652,11 @@ private boolean shouldFireLeaderChange(LeaderAndEpoch leaderAndEpoch) {

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-16 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1296567527 ## metadata/src/main/java/org/apache/kafka/image/node/DelegationTokenDataNode.java: ## Review Comment: There are unit tests to check the redaction state. See

[jira] [Created] (KAFKA-15370) Support Participation in 2PC (KIP-939)

2023-08-16 Thread Artem Livshits (Jira)
Artem Livshits created KAFKA-15370: -- Summary: Support Participation in 2PC (KIP-939) Key: KAFKA-15370 URL: https://issues.apache.org/jira/browse/KAFKA-15370 Project: Kafka Issue Type:

[GitHub] [kafka] artemlivshits opened a new pull request, #14231: [DRAFT] EXAMPLE: dual write recipe example for KIP-939

2023-08-16 Thread via GitHub
artemlivshits opened a new pull request, #14231: URL: https://github.com/apache/kafka/pull/14231 This example uses JDBC and KafkaProducer to implement atomic dual writes to a SQL database and Kafka. The code assumes that KIP-890 part 2 is also implemented, so Kafka commit would bump the

[GitHub] [kafka] gharris1727 opened a new pull request, #14230: KAFKA-15226: Add connect-plugin-path and plugin.discovery system test

2023-08-16 Thread via GitHub
gharris1727 opened a new pull request, #14230: URL: https://github.com/apache/kafka/pull/14230 This test uses a standalone worker to verify the behavior of the plugin.discovery configuration values with non-migrated and migrated artifacts. For a source of non-migrated artifacts, I'm

[jira] [Commented] (KAFKA-15302) Stale value returned when using store.all() in punctuation function.

2023-08-16 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755298#comment-17755298 ] Matthias J. Sax commented on KAFKA-15302: - I would need to think more about this. Overall, it

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-16 Thread via GitHub
cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1296512550 ## metadata/src/main/java/org/apache/kafka/controller/OffsetControlManager.java: ## @@ -212,10 +213,14 @@ long lastStableOffset() { } /** - * @return

[GitHub] [kafka] tanay27 opened a new pull request, #14229: KAFKA-15918: automating version change

2023-08-16 Thread via GitHub
tanay27 opened a new pull request, #14229: URL: https://github.com/apache/kafka/pull/14229 Created a new folder called `release` and moved `release.py`, `release_notes.py` and `reviewers.py` to the new folder. Created two new files: - `requirements.txt` to support all the

[jira] [Resolved] (KAFKA-15220) KRaftMetadataCache returns fenced brokers from getAliveBrokerNode

2023-08-16 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe resolved KAFKA-15220. -- Fix Version/s: 3.6.0 Resolution: Fixed > KRaftMetadataCache returns fenced brokers

[jira] [Created] (KAFKA-15369) Allow AdminClient to Talk Directly with the KRaft Controller Quorum and add Controller Registration

2023-08-16 Thread Colin McCabe (Jira)
Colin McCabe created KAFKA-15369: Summary: Allow AdminClient to Talk Directly with the KRaft Controller Quorum and add Controller Registration Key: KAFKA-15369 URL:

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-16 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1296410110 ## metadata/src/main/java/org/apache/kafka/controller/DelegationTokenControlManager.java: ## @@ -0,0 +1,359 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-16 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1296409238 ## metadata/src/main/java/org/apache/kafka/controller/DelegationTokenControlManager.java: ## @@ -0,0 +1,359 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-16 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1296399634 ## metadata/src/main/java/org/apache/kafka/controller/DelegationTokenControlManager.java: ## @@ -0,0 +1,359 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-16 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1296382585 ## metadata/src/main/java/org/apache/kafka/controller/DelegationTokenControlManager.java: ## @@ -0,0 +1,359 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-16 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1296379603 ## metadata/src/main/java/org/apache/kafka/controller/DelegationTokenControlManager.java: ## @@ -0,0 +1,359 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-16 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1296376127 ## metadata/src/main/java/org/apache/kafka/controller/DelegationTokenControlManager.java: ## @@ -0,0 +1,359 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] jeqo commented on a diff in pull request #12637: KAFKA-14226 (KIP-821): [connect:transform] Introduce support for nested structures

2023-08-16 Thread via GitHub
jeqo commented on code in PR #12637: URL: https://github.com/apache/kafka/pull/12637#discussion_r1296374731 ## connect/transforms/src/main/java/org/apache/kafka/connect/transforms/field/FieldPath.java: ## @@ -0,0 +1,232 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] kamalcph commented on pull request #14116: KAFKA-15167: Tiered Storage Test Harness Framework

2023-08-16 Thread via GitHub
kamalcph commented on PR #14116: URL: https://github.com/apache/kafka/pull/14116#issuecomment-1681198112 @showuon Call for review. PTAL. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [kafka] kamalcph commented on pull request #13947: KAFKA-15130: Delete remote segments when delete a topic

2023-08-16 Thread via GitHub
kamalcph commented on PR #13947: URL: https://github.com/apache/kafka/pull/13947#issuecomment-1681195520 @divijvaidya @junrao @showuon @satishd Addressed your review comments. PTAL. -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [kafka] kamalcph commented on a diff in pull request #13947: KAFKA-15130: Delete remote segments when delete a topic

2023-08-16 Thread via GitHub
kamalcph commented on code in PR #13947: URL: https://github.com/apache/kafka/pull/13947#discussion_r1296367515 ## core/src/test/scala/integration/kafka/admin/RemoteTopicDeletionTest.scala: ## @@ -0,0 +1,148 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [kafka] kamalcph commented on a diff in pull request #13947: KAFKA-15130: Delete remote segments when delete a topic

2023-08-16 Thread via GitHub
kamalcph commented on code in PR #13947: URL: https://github.com/apache/kafka/pull/13947#discussion_r1296367515 ## core/src/test/scala/integration/kafka/admin/RemoteTopicDeletionTest.scala: ## @@ -0,0 +1,148 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one

[GitHub] [kafka] kamalcph commented on a diff in pull request #13947: KAFKA-15130: Delete remote segments when delete a topic

2023-08-16 Thread via GitHub
kamalcph commented on code in PR #13947: URL: https://github.com/apache/kafka/pull/13947#discussion_r1296359728 ## storage/src/main/java/org/apache/kafka/storage/internals/log/RemoteIndexCache.java: ## @@ -166,6 +166,10 @@ public Cache internalCache() { return

[GitHub] [kafka] kamalcph commented on a diff in pull request #13947: KAFKA-15130: Delete remote segments when delete a topic

2023-08-16 Thread via GitHub
kamalcph commented on code in PR #13947: URL: https://github.com/apache/kafka/pull/13947#discussion_r1296357957 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -343,21 +345,80 @@ public void onLeadershipChange(Set partitionsBecomeLeader, /** *

[GitHub] [kafka] kamalcph commented on a diff in pull request #13947: KAFKA-15130: Delete remote segments when delete a topic

2023-08-16 Thread via GitHub
kamalcph commented on code in PR #13947: URL: https://github.com/apache/kafka/pull/13947#discussion_r1296356436 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -343,21 +345,80 @@ public void onLeadershipChange(Set partitionsBecomeLeader, /** *

[GitHub] [kafka] kamalcph commented on a diff in pull request #13947: KAFKA-15130: Delete remote segments when delete a topic

2023-08-16 Thread via GitHub
kamalcph commented on code in PR #13947: URL: https://github.com/apache/kafka/pull/13947#discussion_r1296356436 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -343,21 +345,80 @@ public void onLeadershipChange(Set partitionsBecomeLeader, /** *

[GitHub] [kafka] kamalcph commented on a diff in pull request #13947: KAFKA-15130: Delete remote segments when delete a topic

2023-08-16 Thread via GitHub
kamalcph commented on code in PR #13947: URL: https://github.com/apache/kafka/pull/13947#discussion_r1296353494 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -343,21 +345,80 @@ public void onLeadershipChange(Set partitionsBecomeLeader, /** *

[GitHub] [kafka] kamalcph commented on a diff in pull request #13947: KAFKA-15130: Delete remote segments when delete a topic

2023-08-16 Thread via GitHub
kamalcph commented on code in PR #13947: URL: https://github.com/apache/kafka/pull/13947#discussion_r1296354652 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -343,21 +345,80 @@ public void onLeadershipChange(Set partitionsBecomeLeader, /** *

[GitHub] [kafka] kamalcph commented on a diff in pull request #13947: KAFKA-15130: Delete remote segments when delete a topic

2023-08-16 Thread via GitHub
kamalcph commented on code in PR #13947: URL: https://github.com/apache/kafka/pull/13947#discussion_r1296353850 ## storage/src/main/java/org/apache/kafka/storage/internals/log/RemoteIndexCache.java: ## @@ -421,6 +425,7 @@ public void close() { // Initiate

[GitHub] [kafka] kamalcph commented on a diff in pull request #13947: KAFKA-15130: Delete remote segments when delete a topic

2023-08-16 Thread via GitHub
kamalcph commented on code in PR #13947: URL: https://github.com/apache/kafka/pull/13947#discussion_r1296353494 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -343,21 +345,80 @@ public void onLeadershipChange(Set partitionsBecomeLeader, /** *

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-16 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1296352968 ## metadata/src/main/java/org/apache/kafka/controller/DelegationTokenControlManager.java: ## @@ -0,0 +1,359 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-16 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1296349986 ## metadata/src/main/java/org/apache/kafka/controller/DelegationTokenControlManager.java: ## @@ -0,0 +1,359 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] hgeraldino commented on pull request #14093: KAFKA-15248 Add BooleanConverter

2023-08-16 Thread via GitHub
hgeraldino commented on PR #14093: URL: https://github.com/apache/kafka/pull/14093#issuecomment-1681170703 > Thanks @hgeraldino, LGTM! I just had a few more minor suggestions. Addressed the second round of comments. Thanks again! -- This is an automated message from the Apache Git

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #14182: KAFKA 14515: Optimized Uniform Rack Aware Assignor

2023-08-16 Thread via GitHub
jeffkbkim commented on code in PR #14182: URL: https://github.com/apache/kafka/pull/14182#discussion_r1296343128 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/OptimizedUniformAssignmentBuilder.java: ## @@ -0,0 +1,399 @@ +/* + * Licensed to the

[GitHub] [kafka] rondagostino commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-16 Thread via GitHub
rondagostino commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1296233969 ## metadata/src/main/java/org/apache/kafka/metadata/DelegationTokenData.java: ## Review Comment: Is a Unit test necessary, do you think? I also realize we

[jira] [Resolved] (KAFKA-15228) Add sync-manifests subcommand to connect-plugin-path tool

2023-08-16 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Harris resolved KAFKA-15228. - Resolution: Fixed > Add sync-manifests subcommand to connect-plugin-path tool >

[GitHub] [kafka] gharris1727 merged pull request #14195: KAFKA-15228: Add sync-manifests command to connect-plugin-path (KIP-898)

2023-08-16 Thread via GitHub
gharris1727 merged PR #14195: URL: https://github.com/apache/kafka/pull/14195 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] bachmanity1 commented on pull request #14228: MINOR: Fix ReplicationBytes(Out|In)PerSec metrics' docs

2023-08-16 Thread via GitHub
bachmanity1 commented on PR #14228: URL: https://github.com/apache/kafka/pull/14228#issuecomment-1681087535 @mimaison @ijuma @dongjinleekr can you please review this? Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [kafka] bachmanity1 opened a new pull request, #14228: MINOR: Fix ReplicationBytes(Out|In)PerSec metrics' docs

2023-08-16 Thread via GitHub
bachmanity1 opened a new pull request, #14228: URL: https://github.com/apache/kafka/pull/14228 Based on the code shown below it looks like `ReplicationBytes(Out|In)PerSec` metrics are reported only across all topics and NOT for each individual topic.

[GitHub] [kafka] ijuma commented on pull request #14011: MINOR: Fix MiniKdc Java 17 issue in system tests

2023-08-16 Thread via GitHub
ijuma commented on PR #14011: URL: https://github.com/apache/kafka/pull/14011#issuecomment-1681073825 @gharris1727 you're right, this option should only be provided if the jdk is 9 or newer. My bad for missing that. -- This is an automated message from the Apache Git Service. To respond

[GitHub] [kafka] divijvaidya commented on a diff in pull request #13561: KAFKA-14888: Added remote log segments retention functionality based on time and size.

2023-08-16 Thread via GitHub
divijvaidya commented on code in PR #13561: URL: https://github.com/apache/kafka/pull/13561#discussion_r1296264710 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -696,11 +704,327 @@ public void run() { } } +public void

[GitHub] [kafka] divijvaidya commented on a diff in pull request #13561: KAFKA-14888: Added remote log segments retention functionality based on time and size.

2023-08-16 Thread via GitHub
divijvaidya commented on code in PR #13561: URL: https://github.com/apache/kafka/pull/13561#discussion_r1296264710 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -696,11 +704,327 @@ public void run() { } } +public void

[jira] [Commented] (KAFKA-15223) Need more clarity in documentation for upgrade/downgrade procedures and limitations across releases.

2023-08-16 Thread kaushik srinivas (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755207#comment-17755207 ] kaushik srinivas commented on KAFKA-15223: -- Any updates on this ? > Need more clarity in

[GitHub] [kafka] cadonna commented on a diff in pull request #13927: [DO NOT MERGE] KAFKA-10199: Enable state updater by default

2023-08-16 Thread via GitHub
cadonna commented on code in PR #13927: URL: https://github.com/apache/kafka/pull/13927#discussion_r1296232439 ## streams/src/test/java/org/apache/kafka/streams/integration/PurgeRepartitionTopicIntegrationTest.java: ## @@ -212,10 +213,11 @@ public void shouldRestoreState()

[GitHub] [kafka] mumrah commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-16 Thread via GitHub
mumrah commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1296231483 ## metadata/src/main/java/org/apache/kafka/controller/OffsetControlManager.java: ## @@ -212,10 +213,14 @@ long lastStableOffset() { } /** - * @return

[GitHub] [kafka] gharris1727 merged pull request #13302: KAFKA-14759: Move Mock, Schema, and Verifiable connectors to new test-plugins module

2023-08-16 Thread via GitHub
gharris1727 merged PR #13302: URL: https://github.com/apache/kafka/pull/13302 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] gharris1727 commented on pull request #13302: KAFKA-14759: Move Mock, Schema, and Verifiable connectors to new test-plugins module

2023-08-16 Thread via GitHub
gharris1727 commented on PR #13302: URL: https://github.com/apache/kafka/pull/13302#issuecomment-1681010261 Test failures in CI appear unrelated. I ran the Connect system tests for this change with #14011 reverted and they all passed. -- This is an automated message from the Apache Git

[GitHub] [kafka] gharris1727 commented on pull request #14220: KAFKA-15102: Add release notes about the replication.policy.internal.topic.separator.enabled property for MirrorMaker 2

2023-08-16 Thread via GitHub
gharris1727 commented on PR #14220: URL: https://github.com/apache/kafka/pull/14220#issuecomment-1681003739 > I'd just cherry-pick the commit for this PR, but we don't have autogenerated docs for the MM2 properties on those branches, so the wording will have to be different. Yes the

[GitHub] [kafka] junrao commented on a diff in pull request #13561: KAFKA-14888: Added remote log segments retention functionality based on time and size.

2023-08-16 Thread via GitHub
junrao commented on code in PR #13561: URL: https://github.com/apache/kafka/pull/13561#discussion_r1296221075 ## core/src/main/scala/kafka/log/UnifiedLog.scala: ## @@ -1390,7 +1424,15 @@ class UnifiedLog(@volatile var logStartOffset: Long, private def

[GitHub] [kafka] mumrah commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-16 Thread via GitHub
mumrah commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1296220658 ## metadata/src/main/java/org/apache/kafka/controller/QuorumController.java: ## @@ -892,48 +915,73 @@ class MigrationWriteOperation implements ControllerWriteOperation

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-16 Thread via GitHub
cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1296172460 ## metadata/src/main/java/org/apache/kafka/controller/QuorumController.java: ## @@ -892,48 +915,73 @@ class MigrationWriteOperation implements

[GitHub] [kafka] cadonna opened a new pull request, #14227: MINOR: Decouple purging committed records from committing

2023-08-16 Thread via GitHub
cadonna opened a new pull request, #14227: URL: https://github.com/apache/kafka/pull/14227 Currently, Kafka Streams only tries to purge records whose offset are committed from a repartition topic when at least one offset was committed in the current commit. The coupling between committing

[GitHub] [kafka] lucasbru opened a new pull request, #14226: Kafka Streams Threading: Timeout behavior

2023-08-16 Thread via GitHub
lucasbru opened a new pull request, #14226: URL: https://github.com/apache/kafka/pull/14226 The process method inside the tasks needs to be called from within the processing threads. However, it currently interacts with the consumer in two ways: - It resumes processing when the

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-16 Thread via GitHub
cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1296200794 ## metadata/src/main/java/org/apache/kafka/image/loader/LogDeltaManifest.java: ## @@ -66,6 +119,10 @@ public LogDeltaManifest( this.numBytes = numBytes;

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-16 Thread via GitHub
cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r129629 ## metadata/src/main/java/org/apache/kafka/controller/OffsetControlManager.java: ## @@ -212,10 +213,14 @@ long lastStableOffset() { } /** - * @return

[jira] [Commented] (KAFKA-15338) The metric group documentation for metrics added in KAFKA-13945 is incorrect

2023-08-16 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755184#comment-17755184 ] Matthias J. Sax commented on KAFKA-15338: - Sure. Thank a lot! > The metric group documentation

[GitHub] [kafka] gharris1727 commented on a diff in pull request #14220: KAFKA-15102: Add release notes about the replication.policy.internal.topic.separator.enabled property for MirrorMaker 2

2023-08-16 Thread via GitHub
gharris1727 commented on code in PR #14220: URL: https://github.com/apache/kafka/pull/14220#discussion_r1296189852 ## docs/upgrade.html: ## @@ -43,6 +43,10 @@ Notable changes in 3 See

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-16 Thread via GitHub
cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1296178891 ## metadata/src/main/java/org/apache/kafka/controller/QuorumController.java: ## @@ -892,48 +915,73 @@ class MigrationWriteOperation implements

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-16 Thread via GitHub
cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1296178891 ## metadata/src/main/java/org/apache/kafka/controller/QuorumController.java: ## @@ -892,48 +915,73 @@ class MigrationWriteOperation implements

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-16 Thread via GitHub
cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1296178891 ## metadata/src/main/java/org/apache/kafka/controller/QuorumController.java: ## @@ -892,48 +915,73 @@ class MigrationWriteOperation implements

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-16 Thread via GitHub
cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1296173796 ## metadata/src/main/java/org/apache/kafka/controller/QuorumController.java: ## @@ -892,48 +915,73 @@ class MigrationWriteOperation implements

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-16 Thread via GitHub
cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1296172460 ## metadata/src/main/java/org/apache/kafka/controller/QuorumController.java: ## @@ -892,48 +915,73 @@ class MigrationWriteOperation implements

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-16 Thread via GitHub
cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1296173796 ## metadata/src/main/java/org/apache/kafka/controller/QuorumController.java: ## @@ -892,48 +915,73 @@ class MigrationWriteOperation implements

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-16 Thread via GitHub
cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1296172460 ## metadata/src/main/java/org/apache/kafka/controller/QuorumController.java: ## @@ -892,48 +915,73 @@ class MigrationWriteOperation implements

[GitHub] [kafka] kamalcph commented on a diff in pull request #14136: Add metadatacache into RemoteLogManager, and refactor all relevant codes

2023-08-16 Thread via GitHub
kamalcph commented on code in PR #14136: URL: https://github.com/apache/kafka/pull/14136#discussion_r1296153252 ## core/src/test/java/kafka/log/remote/RemoteLogManagerTest.java: ## @@ -802,41 +817,25 @@ private void verifyInCache(TopicIdPartition... topicIdPartitions) {

[GitHub] [kafka] nikramakrishnan commented on a diff in pull request #14212: MINOR: Add test case for follower fetch

2023-08-16 Thread via GitHub
nikramakrishnan commented on code in PR #14212: URL: https://github.com/apache/kafka/pull/14212#discussion_r1296157690 ## core/src/test/scala/unit/kafka/server/ReplicaFetcherTierStateMachineTest.scala: ## @@ -85,6 +85,73 @@ class ReplicaFetcherTierStateMachineTest {

[GitHub] [kafka] nikramakrishnan commented on a diff in pull request #14212: MINOR: Add test case for follower fetch

2023-08-16 Thread via GitHub
nikramakrishnan commented on code in PR #14212: URL: https://github.com/apache/kafka/pull/14212#discussion_r1295970964 ## core/src/test/scala/unit/kafka/server/ReplicaFetcherTierStateMachineTest.scala: ## @@ -85,6 +85,64 @@ class ReplicaFetcherTierStateMachineTest {

[GitHub] [kafka] kamalcph commented on a diff in pull request #14161: KAFKA-15267: Do not allow Tiered Storage to be disabled while topics have remote.storage.enable property

2023-08-16 Thread via GitHub
kamalcph commented on code in PR #14161: URL: https://github.com/apache/kafka/pull/14161#discussion_r1296143313 ## core/src/test/scala/unit/kafka/server/KafkaServerTest.scala: ## @@ -154,6 +155,96 @@ class KafkaServerTest extends QuorumTestHarness { server.shutdown() }

[GitHub] [kafka] C0urante commented on pull request #14220: KAFKA-15102: Add release notes about the replication.policy.internal.topic.separator.enabled property for MirrorMaker 2

2023-08-16 Thread via GitHub
C0urante commented on PR #14220: URL: https://github.com/apache/kafka/pull/14220#issuecomment-1680894253 > I think that we should copy this line item to the 3.1.x-3.5.x documentation when the patch release for those versions go out, but we might be in danger of forgetting. Can you create

[jira] [Commented] (KAFKA-14588) Move ConfigCommand to tools

2023-08-16 Thread Nikolay Izhikov (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755164#comment-17755164 ] Nikolay Izhikov commented on KAFKA-14588: - Hello [~omnia_h_ibrahim]. Can I assign this ticket to

[jira] [Commented] (KAFKA-14589) Move ConsumerGroupCommand to tools

2023-08-16 Thread Nikolay Izhikov (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755163#comment-17755163 ] Nikolay Izhikov commented on KAFKA-14589: - Hello [~omnia_h_ibrahim]. Can I assign this ticket to

[GitHub] [kafka] gharris1727 commented on pull request #14011: MINOR: Fix MiniKdc Java 17 issue in system tests

2023-08-16 Thread via GitHub
gharris1727 commented on PR #14011: URL: https://github.com/apache/kafka/pull/14011#issuecomment-1680884808 Hi @see-quick @ijuma @showuon I'm having issues running system tests with this change. I use the default settings, as recommended in the README: ``` TC_PATHS="." bash

[GitHub] [kafka] nizhikov commented on pull request #13247: KAFKA-14595 Move ReassignPartitionsCommand to java

2023-08-16 Thread via GitHub
nizhikov commented on PR #13247: URL: https://github.com/apache/kafka/pull/13247#issuecomment-1680882516 It seems tests are OK in #14217 Please, review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

  1   2   >