[GitHub] [kafka] kamalcph commented on pull request #14176: KAFKA-15295: Add config validation when remote storage is enabled on a topic

2023-08-14 Thread via GitHub
kamalcph commented on PR #14176: URL: https://github.com/apache/kafka/pull/14176#issuecomment-1678466836 @showuon @clolov Addressed your review comments. PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[jira] [Created] (KAFKA-15343) Fix MirrorConnectIntegrationTests causing ci build failures.

2023-08-14 Thread Prasanth Kumar (Jira)
Prasanth Kumar created KAFKA-15343: -- Summary: Fix MirrorConnectIntegrationTests causing ci build failures. Key: KAFKA-15343 URL: https://issues.apache.org/jira/browse/KAFKA-15343 Project: Kafka

[GitHub] [kafka] showuon commented on a diff in pull request #14202: KAFKA-15329: Make default remote.log.metadata.manager.class.name as topic based RLMM

2023-08-14 Thread via GitHub
showuon commented on code in PR #14202: URL: https://github.com/apache/kafka/pull/14202#discussion_r1294184526 ## storage/src/test/java/org/apache/kafka/server/log/remote/storage/RemoteLogManagerConfigTest.java: ## @@ -52,6 +54,28 @@ public void testValidConfigs() { Ass

[GitHub] [kafka] showuon commented on a diff in pull request #14176: KAFKA-15295: Add config validation when remote storage is enabled on a topic

2023-08-14 Thread via GitHub
showuon commented on code in PR #14176: URL: https://github.com/apache/kafka/pull/14176#discussion_r1294181153 ## storage/src/main/java/org/apache/kafka/storage/internals/log/LogConfig.java: ## @@ -265,7 +266,12 @@ public Optional serverConfigName(String configName) {

[GitHub] [kafka] showuon commented on a diff in pull request #14176: KAFKA-15295: Add config validation when remote storage is enabled on a topic

2023-08-14 Thread via GitHub
showuon commented on code in PR #14176: URL: https://github.com/apache/kafka/pull/14176#discussion_r1294181153 ## storage/src/main/java/org/apache/kafka/storage/internals/log/LogConfig.java: ## @@ -265,7 +266,12 @@ public Optional serverConfigName(String configName) {

[GitHub] [kafka] kamalcph commented on a diff in pull request #14176: KAFKA-15295: Add config validation when remote storage is enabled on a topic

2023-08-14 Thread via GitHub
kamalcph commented on code in PR #14176: URL: https://github.com/apache/kafka/pull/14176#discussion_r1294167123 ## storage/src/main/java/org/apache/kafka/storage/internals/log/LogConfig.java: ## @@ -265,7 +266,12 @@ public Optional serverConfigName(String configName) {

[GitHub] [kafka] showuon commented on a diff in pull request #14161: KAFKA-15267: Do not allow Tiered Storage to be disabled while topics have remote.storage.enable property

2023-08-14 Thread via GitHub
showuon commented on code in PR #14161: URL: https://github.com/apache/kafka/pull/14161#discussion_r1294147126 ## core/src/main/scala/kafka/server/ConfigHandler.scala: ## @@ -62,6 +62,12 @@ class TopicConfigHandler(private val logManager: LogManager, kafkaConfig: KafkaC to

[GitHub] [kafka] showuon commented on a diff in pull request #14161: KAFKA-15267: Do not allow Tiered Storage to be disabled while topics have remote.storage.enable property

2023-08-14 Thread via GitHub
showuon commented on code in PR #14161: URL: https://github.com/apache/kafka/pull/14161#discussion_r1294147126 ## core/src/main/scala/kafka/server/ConfigHandler.scala: ## @@ -62,6 +62,12 @@ class TopicConfigHandler(private val logManager: LogManager, kafkaConfig: KafkaC to

[GitHub] [kafka] showuon commented on a diff in pull request #14176: KAFKA-15295: Add config validation when remote storage is enabled on a topic

2023-08-14 Thread via GitHub
showuon commented on code in PR #14176: URL: https://github.com/apache/kafka/pull/14176#discussion_r1294139696 ## storage/src/main/java/org/apache/kafka/storage/internals/log/LogConfig.java: ## @@ -454,54 +460,101 @@ public static void validateNames(Properties props) {

[GitHub] [kafka] showuon commented on pull request #14176: KAFKA-15295: Add config validation when remote storage is enabled on a topic

2023-08-14 Thread via GitHub
showuon commented on PR #14176: URL: https://github.com/apache/kafka/pull/14176#issuecomment-1678365110 > Please correct me if I have understood this wrongly, but this pull request proposes propagating a configuration known only to the RemoteLogManagerConfig to the LogConfig via the KafkaCo

[GitHub] [kafka] kamalcph commented on a diff in pull request #13947: KAFKA-15130: Delete remote segments when delete a topic

2023-08-14 Thread via GitHub
kamalcph commented on code in PR #13947: URL: https://github.com/apache/kafka/pull/13947#discussion_r1294097096 ## core/src/test/scala/integration/kafka/admin/RemoteTopicCRUDTest.scala: ## @@ -0,0 +1,148 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or mo

[GitHub] [kafka] mumrah opened a new pull request, #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-14 Thread via GitHub
mumrah opened a new pull request, #14208: URL: https://github.com/apache/kafka/pull/14208 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mai

[GitHub] [kafka] jsancio commented on pull request #14168: MINOR; Fix nanosecond elapsed time

2023-08-14 Thread via GitHub
jsancio commented on PR #14168: URL: https://github.com/apache/kafka/pull/14168#issuecomment-1678276370 > `testBalancePartitionLeaders` is still failing in the CI after this change. Can you please check that? Correct. I'll try to spend sometime this week to figure out why it is faili

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #14182: KAFKA 14515: Optimized Uniform Rack Aware Assignor

2023-08-14 Thread via GitHub
jeffkbkim commented on code in PR #14182: URL: https://github.com/apache/kafka/pull/14182#discussion_r1291721804 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/AbstractUniformAssignor.java: ## @@ -0,0 +1,220 @@ +/* + * Licensed to the Apache Softw

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #14182: KAFKA 14515: Optimized Uniform Rack Aware Assignor

2023-08-14 Thread via GitHub
jeffkbkim commented on code in PR #14182: URL: https://github.com/apache/kafka/pull/14182#discussion_r1291714013 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/AbstractUniformAssignor.java: ## @@ -0,0 +1,220 @@ +/* + * Licensed to the Apache Softw

[GitHub] [kafka] gharris1727 commented on a diff in pull request #14195: KAFKA-15228: Add sync-manifests command to connect-plugin-path (KIP-898)

2023-08-14 Thread via GitHub
gharris1727 commented on code in PR #14195: URL: https://github.com/apache/kafka/pull/14195#discussion_r1294083162 ## tools/src/main/java/org/apache/kafka/tools/ConnectPluginPath.java: ## @@ -368,6 +391,30 @@ private static void endCommand( config.out.printf("Total

[GitHub] [kafka] cmccabe commented on pull request #13033: KAFKA-14538: Implement KRaft metadata transactions

2023-08-14 Thread via GitHub
cmccabe commented on PR #13033: URL: https://github.com/apache/kafka/pull/13033#issuecomment-1678247367 superseded by #14200 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [kafka] cmccabe closed pull request #13033: KAFKA-14538: Implement KRaft metadata transactions

2023-08-14 Thread via GitHub
cmccabe closed pull request #13033: KAFKA-14538: Implement KRaft metadata transactions URL: https://github.com/apache/kafka/pull/13033 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[GitHub] [kafka] cmccabe closed pull request #14132: KAFKA-14538: Implement KRaft metadata transactions

2023-08-14 Thread via GitHub
cmccabe closed pull request #14132: KAFKA-14538: Implement KRaft metadata transactions URL: https://github.com/apache/kafka/pull/14132 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[GitHub] [kafka] cmccabe commented on pull request #14132: KAFKA-14538: Implement KRaft metadata transactions

2023-08-14 Thread via GitHub
cmccabe commented on PR #14132: URL: https://github.com/apache/kafka/pull/14132#issuecomment-1678246884 duplicate of #14200 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [kafka] cmccabe merged pull request #14200: KAFKA-14538: Implement KRaft metadata transactions in QuorumController

2023-08-14 Thread via GitHub
cmccabe merged PR #14200: URL: https://github.com/apache/kafka/pull/14200 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[GitHub] [kafka] gharris1727 commented on a diff in pull request #14195: KAFKA-15228: Add sync-manifests command to connect-plugin-path (KIP-898)

2023-08-14 Thread via GitHub
gharris1727 commented on code in PR #14195: URL: https://github.com/apache/kafka/pull/14195#discussion_r1294058501 ## tools/src/main/java/org/apache/kafka/tools/ManifestWorkspace.java: ## @@ -0,0 +1,573 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or mor

[GitHub] [kafka] gharris1727 commented on a diff in pull request #14195: KAFKA-15228: Add sync-manifests command to connect-plugin-path (KIP-898)

2023-08-14 Thread via GitHub
gharris1727 commented on code in PR #14195: URL: https://github.com/apache/kafka/pull/14195#discussion_r1294055531 ## tools/src/main/java/org/apache/kafka/tools/ManifestWorkspace.java: ## @@ -0,0 +1,573 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or mor

[GitHub] [kafka] gharris1727 commented on a diff in pull request #14195: KAFKA-15228: Add sync-manifests command to connect-plugin-path (KIP-898)

2023-08-14 Thread via GitHub
gharris1727 commented on code in PR #14195: URL: https://github.com/apache/kafka/pull/14195#discussion_r1294055068 ## tools/src/test/java/org/apache/kafka/tools/ConnectPluginPathTest.java: ## @@ -192,6 +194,51 @@ public void testListMultipleWorkerConfigs(PluginLocationType type

[GitHub] [kafka] gharris1727 commented on a diff in pull request #14195: KAFKA-15228: Add sync-manifests command to connect-plugin-path (KIP-898)

2023-08-14 Thread via GitHub
gharris1727 commented on code in PR #14195: URL: https://github.com/apache/kafka/pull/14195#discussion_r1294053643 ## tools/src/main/java/org/apache/kafka/tools/ManifestWorkspace.java: ## @@ -0,0 +1,573 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or mor

[GitHub] [kafka] gharris1727 commented on a diff in pull request #14195: KAFKA-15228: Add sync-manifests command to connect-plugin-path (KIP-898)

2023-08-14 Thread via GitHub
gharris1727 commented on code in PR #14195: URL: https://github.com/apache/kafka/pull/14195#discussion_r1294053484 ## tools/src/main/java/org/apache/kafka/tools/ManifestWorkspace.java: ## @@ -0,0 +1,573 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or mor

[GitHub] [kafka] Nickstery commented on a diff in pull request #14104: Tiered storage | Segment with corrupted index should not be uploaded to remote storage

2023-08-14 Thread via GitHub
Nickstery commented on code in PR #14104: URL: https://github.com/apache/kafka/pull/14104#discussion_r1294052739 ## storage/src/main/java/org/apache/kafka/storage/internals/log/TimeIndex.java: ## @@ -72,15 +72,17 @@ public TimeIndex(File file, long baseOffset, int maxIndexSize,

[GitHub] [kafka] jolshan merged pull request #14138: Fix a race when query isUnderMinIsr

2023-08-14 Thread via GitHub
jolshan merged PR #14138: URL: https://github.com/apache/kafka/pull/14138 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[jira] [Assigned] (KAFKA-15308) Wipe Stores upon OffsetOutOfRangeException in ALOS

2023-08-14 Thread A. Sophie Blee-Goldman (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] A. Sophie Blee-Goldman reassigned KAFKA-15308: -- Assignee: Rohan Desai > Wipe Stores upon OffsetOutOfRangeException in

[GitHub] [kafka] jeqo commented on a diff in pull request #14127: KAFKA-15181: Wait for RemoteLogMetadataCache to initialize after assigning partitions

2023-08-14 Thread via GitHub
jeqo commented on code in PR #14127: URL: https://github.com/apache/kafka/pull/14127#discussion_r1294016456 ## storage/src/main/java/org/apache/kafka/server/log/remote/metadata/storage/ConsumerTask.java: ## @@ -64,302 +64,395 @@ class ConsumerTask implements Runnable, Closeable

[GitHub] [kafka] ethaden commented on a diff in pull request #14130: KAFKA-15273: Log common name of expired client certificates

2023-08-14 Thread via GitHub
ethaden commented on code in PR #14130: URL: https://github.com/apache/kafka/pull/14130#discussion_r1294020659 ## clients/src/test/resources/log4j.properties: ## @@ -19,3 +19,5 @@ log4j.appender.stdout.layout=org.apache.log4j.PatternLayout log4j.appender.stdout.layout.Conversio

[GitHub] [kafka] ethaden commented on a diff in pull request #14130: KAFKA-15273: Log common name of expired client certificates

2023-08-14 Thread via GitHub
ethaden commented on code in PR #14130: URL: https://github.com/apache/kafka/pull/14130#discussion_r1294016888 ## clients/src/main/java/org/apache/kafka/common/security/ssl/CommonNameLoggingTrustManagerFactoryWrapper.java: ## @@ -0,0 +1,423 @@ +/* + * Licensed to the Apache Soft

[GitHub] [kafka] junrao commented on a diff in pull request #13947: KAFKA-15130: Delete remote segments when delete a topic

2023-08-14 Thread via GitHub
junrao commented on code in PR #13947: URL: https://github.com/apache/kafka/pull/13947#discussion_r1294012031 ## core/src/test/scala/integration/kafka/admin/RemoteTopicCRUDTest.scala: ## @@ -0,0 +1,148 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more

[GitHub] [kafka] cmccabe commented on a diff in pull request #14200: KAFKA-14538: Implement KRaft metadata transactions in QuorumController

2023-08-14 Thread via GitHub
cmccabe commented on code in PR #14200: URL: https://github.com/apache/kafka/pull/14200#discussion_r1294013056 ## metadata/src/main/resources/common/metadata/AbortTransactionRecord.json: ## @@ -0,0 +1,24 @@ +// Licensed to the Apache Software Foundation (ASF) under one or more +

[GitHub] [kafka] rondagostino commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-14 Thread via GitHub
rondagostino commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1293962499 ## metadata/src/test/java/org/apache/kafka/image/DelegationTokenImageTest.java: ## Review Comment: Can we include `RemoveDelegationTokenRecord` in a test?

[GitHub] [kafka] gharris1727 commented on a diff in pull request #14195: KAFKA-15228: Add sync-manifests command to connect-plugin-path (KIP-898)

2023-08-14 Thread via GitHub
gharris1727 commented on code in PR #14195: URL: https://github.com/apache/kafka/pull/14195#discussion_r1293958574 ## tools/src/main/java/org/apache/kafka/tools/ManifestWorkspace.java: ## @@ -0,0 +1,573 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or mor

[GitHub] [kafka] prestona commented on pull request #14173: KAFKA-15042: Improve TAGGED_FIELDS protocol documentation

2023-08-14 Thread via GitHub
prestona commented on PR #14173: URL: https://github.com/apache/kafka/pull/14173#issuecomment-1678023214 @C0urante / @tombentley (as recent approvers for issues with a `docs` tag), if you have a few moments to look over this one - I'd appreciate your feedback. Thanks. -- This is an autom

[GitHub] [kafka] philipnee opened a new pull request, #14207: Consumer refactor group heartbeat request manager

2023-08-14 Thread via GitHub
philipnee opened a new pull request, #14207: URL: https://github.com/apache/kafka/pull/14207 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of testi

[GitHub] [kafka] jeffkbkim commented on pull request #14124: Kafka-14509; [1/2] Define ConsumerGroupDescribe API request and response schemas and classes.

2023-08-14 Thread via GitHub
jeffkbkim commented on PR #14124: URL: https://github.com/apache/kafka/pull/14124#issuecomment-1677965382 @riedelmax is it possible to make the tests pass i.e. do nothing? then we can make the changes in the second PR -- This is an automated message from the Apache Git Service. To respond

[GitHub] [kafka] cadonna commented on a diff in pull request #13927: [DO NOT MERGE] KAFKA-10199: Enable state updater by default

2023-08-14 Thread via GitHub
cadonna commented on code in PR #13927: URL: https://github.com/apache/kafka/pull/13927#discussion_r1293901249 ## streams/src/test/java/org/apache/kafka/streams/integration/PurgeRepartitionTopicIntegrationTest.java: ## @@ -212,10 +213,11 @@ public void shouldRestoreState() throw

[GitHub] [kafka] C0urante commented on pull request #14005: KAFKA-15177: Implement KIP-875 SourceConnector::alterOffset API in MirrorMaker 2 connectors

2023-08-14 Thread via GitHub
C0urante commented on PR #14005: URL: https://github.com/apache/kafka/pull/14005#issuecomment-1677950337 Thanks @gharris1727! @mimaison If you'd still like to weigh in, I can leave this open until Wednesday (two days from now). -- This is an automated message from the Apache Git Se

[GitHub] [kafka] C0urante commented on a diff in pull request #14195: KAFKA-15228: Add sync-manifests command to connect-plugin-path (KIP-898)

2023-08-14 Thread via GitHub
C0urante commented on code in PR #14195: URL: https://github.com/apache/kafka/pull/14195#discussion_r1293726875 ## tools/src/main/java/org/apache/kafka/tools/ConnectPluginPath.java: ## @@ -368,6 +391,30 @@ private static void endCommand( config.out.printf("Total plu

[GitHub] [kafka] jeffkbkim commented on a diff in pull request #14182: KAFKA 14515: Optimized Uniform Rack Aware Assignor

2023-08-14 Thread via GitHub
jeffkbkim commented on code in PR #14182: URL: https://github.com/apache/kafka/pull/14182#discussion_r1291714013 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/AbstractUniformAssignor.java: ## @@ -0,0 +1,220 @@ +/* + * Licensed to the Apache Softw

[GitHub] [kafka] mumrah commented on a diff in pull request #14200: KAFKA-14538: Implement KRaft metadata transactions in QuorumController

2023-08-14 Thread via GitHub
mumrah commented on code in PR #14200: URL: https://github.com/apache/kafka/pull/14200#discussion_r1293841115 ## metadata/src/main/resources/common/metadata/AbortTransactionRecord.json: ## @@ -0,0 +1,24 @@ +// Licensed to the Apache Software Foundation (ASF) under one or more +/

[GitHub] [kafka] junrao commented on a diff in pull request #13561: KAFKA-14888: Added remote log segments retention functionality based on time and size.

2023-08-14 Thread via GitHub
junrao commented on code in PR #13561: URL: https://github.com/apache/kafka/pull/13561#discussion_r1293851913 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -698,11 +707,329 @@ public void run() { } } +public void handleLogSta

[GitHub] [kafka] junrao commented on a diff in pull request #13561: KAFKA-14888: Added remote log segments retention functionality based on time and size.

2023-08-14 Thread via GitHub
junrao commented on code in PR #13561: URL: https://github.com/apache/kafka/pull/13561#discussion_r1293823449 ## core/src/main/scala/kafka/log/UnifiedLog.scala: ## @@ -2202,19 +2262,24 @@ case class RetentionMsBreach(log: UnifiedLog) extends SegmentDeletionReason { } } -c

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-14 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1293796381 ## metadata/src/main/java/org/apache/kafka/controller/DelegationTokenControlManager.java: ## @@ -0,0 +1,381 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-14 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1293793859 ## metadata/src/main/java/org/apache/kafka/controller/DelegationTokenControlManager.java: ## @@ -0,0 +1,381 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-14 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1293786529 ## core/src/main/scala/kafka/server/ControllerApis.scala: ## @@ -842,6 +847,75 @@ class ControllerApis(val requestChannel: RequestChannel, } } + def ha

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-14 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1293786264 ## core/src/main/scala/kafka/server/DelegationTokenManager.scala: ## @@ -24,30 +24,20 @@ import java.util.Base64 import javax.crypto.spec.SecretKeySpec import j

[GitHub] [kafka] ahuang98 opened a new pull request, #14206: [MINOR] More image replay test cases

2023-08-14 Thread via GitHub
ahuang98 opened a new pull request, #14206: URL: https://github.com/apache/kafka/pull/14206 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgr

[jira] [Updated] (KAFKA-15337) Disable *-subscription-store-changelog topic creation for foreign key join in Kafka Streams

2023-08-14 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15337: Labels: need-kip (was: ) > Disable *-subscription-store-changelog topic creation for fore

[jira] [Updated] (KAFKA-15337) Disable *-subscription-store-changelog topic creation for foreign key join in Kafka Streams

2023-08-14 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15337: Component/s: streams > Disable *-subscription-store-changelog topic creation for foreign k

[GitHub] [kafka] junrao merged pull request #13990: KAFKA-14937: Refactoring for client code to reduce boilerplate

2023-08-14 Thread via GitHub
junrao merged PR #13990: URL: https://github.com/apache/kafka/pull/13990 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.o

[GitHub] [kafka] AndrewJSchofield commented on a diff in pull request #14111: KAFKA-9800: Exponential backoff for Kafka clients - KIP-580

2023-08-14 Thread via GitHub
AndrewJSchofield commented on code in PR #14111: URL: https://github.com/apache/kafka/pull/14111#discussion_r1293735321 ## clients/src/main/java/org/apache/kafka/clients/Metadata.java: ## @@ -81,23 +85,32 @@ public class Metadata implements Closeable { * * @param ref

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-14 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1293675820 ## metadata/src/main/java/org/apache/kafka/controller/DelegationTokenControlManager.java: ## @@ -0,0 +1,381 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] [kafka] cadonna merged pull request #14001: KAFKA-15326: [5/N] Processing thread punctuation

2023-08-14 Thread via GitHub
cadonna merged PR #14001: URL: https://github.com/apache/kafka/pull/14001 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.

[GitHub] [kafka] cadonna commented on pull request #14001: KAFKA-15326: [5/N] Processing thread punctuation

2023-08-14 Thread via GitHub
cadonna commented on PR #14001: URL: https://github.com/apache/kafka/pull/14001#issuecomment-1677529494 Build failures are unrelated: ``` Build / JDK 11 and Scala 2.13 / org.apache.kafka.connect.mirror.integration.MirrorConnectorsIntegrationExactlyOnceTest.testOffsetTranslationBehindRe

[GitHub] [kafka] philipnee commented on pull request #13990: KAFKA-14937: Refactoring for client code to reduce boilerplate

2023-08-14 Thread via GitHub
philipnee commented on PR #13990: URL: https://github.com/apache/kafka/pull/13990#issuecomment-1677501586 Hey @junrao - from the previous runs. it seems like JDK17 has been completing with flaky tests. https://ci-builds.apache.org/blue/organizations/jenkins/Kafka%2Fkafka-pr/activity?branch

[GitHub] [kafka] junrao commented on pull request #13990: KAFKA-14937: Refactoring for client code to reduce boilerplate

2023-08-14 Thread via GitHub
junrao commented on PR #13990: URL: https://github.com/apache/kafka/pull/13990#issuecomment-1677476713 @kirktrue : JDK 17 build was aborted. Did that pass locally? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[GitHub] [kafka] kamalcph commented on a diff in pull request #13947: KAFKA-15130: Delete remote segments when delete a topic

2023-08-14 Thread via GitHub
kamalcph commented on code in PR #13947: URL: https://github.com/apache/kafka/pull/13947#discussion_r1293498028 ## storage/src/main/java/org/apache/kafka/storage/internals/log/RemoteIndexCache.java: ## @@ -421,6 +425,7 @@ public void close() { // Initiate shutdo

[GitHub] [kafka] omkreddy commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-14 Thread via GitHub
omkreddy commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1293480410 ## core/src/main/scala/kafka/server/ControllerApis.scala: ## @@ -842,6 +847,75 @@ class ControllerApis(val requestChannel: RequestChannel, } } + def handl

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-14 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1293465257 ## core/src/main/scala/kafka/server/ControllerApis.scala: ## @@ -842,6 +847,75 @@ class ControllerApis(val requestChannel: RequestChannel, } } + def ha

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-14 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1293458695 ## metadata/src/main/java/org/apache/kafka/controller/QuorumController.java: ## @@ -1498,6 +1543,30 @@ private void cancelNextWriteNoOpRecord() { queue.can

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-14 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1293455737 ## metadata/src/main/java/org/apache/kafka/metadata/migration/KRaftMigrationZkWriter.java: ## @@ -633,4 +641,35 @@ void handleAclsDelta(AclsImage image, AclsDelta d

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-14 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1293448066 ## core/src/test/scala/unit/kafka/server/DelegationTokenRequestsTest.scala: ## @@ -109,13 +127,44 @@ class DelegationTokenRequestsTest extends BaseRequestTest with

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-14 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1293441755 ## metadata/src/main/java/org/apache/kafka/controller/DelegationTokenControlManager.java: ## @@ -0,0 +1,380 @@ +/* + * Licensed to the Apache Software Foundation (A

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-14 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1293439494 ## server-common/src/main/java/org/apache/kafka/server/common/MetadataVersion.java: ## @@ -176,8 +176,11 @@ public enum MetadataVersion { // Support for SCRAM

[GitHub] [kafka] kamalcph commented on pull request #14176: KAFKA-15295: Add config validation when remote storage is enabled on a topic

2023-08-14 Thread via GitHub
kamalcph commented on PR #14176: URL: https://github.com/apache/kafka/pull/14176#issuecomment-1677269388 > Please correct me if I have understood this wrongly, but this pull request proposes propagating a configuration known only to the RemoteLogManagerConfig to the LogConfig via the KafkaC

[GitHub] [kafka] divijvaidya commented on a diff in pull request #13947: KAFKA-15130: Delete remote segments when delete a topic

2023-08-14 Thread via GitHub
divijvaidya commented on code in PR #13947: URL: https://github.com/apache/kafka/pull/13947#discussion_r1292478923 ## storage/api/src/test/java/org/apache/kafka/server/log/remote/storage/NoOpRemoteLogMetadataManager.java: ## @@ -29,12 +29,16 @@ public class NoOpRemoteLogMetadat

[GitHub] [kafka] divijvaidya commented on a diff in pull request #13947: KAFKA-15130: Delete remote segments when delete a topic

2023-08-14 Thread via GitHub
divijvaidya commented on code in PR #13947: URL: https://github.com/apache/kafka/pull/13947#discussion_r1293423318 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -343,21 +345,78 @@ public void onLeadershipChange(Set partitionsBecomeLeader, /** *

[GitHub] [kafka] kamalcph commented on a diff in pull request #14176: KAFKA-15295: Add config validation when remote storage is enabled on a topic

2023-08-14 Thread via GitHub
kamalcph commented on code in PR #14176: URL: https://github.com/apache/kafka/pull/14176#discussion_r1293423526 ## core/src/test/scala/integration/kafka/admin/RemoteTopicCRUDTest.scala: ## @@ -0,0 +1,262 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or m

[jira] [Updated] (KAFKA-15342) Considering upgrading to Mockito 5.4.1 or later

2023-08-14 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton updated KAFKA-15342: -- Description: We're currently stuck on Mockito 4.x.y because the 5.x.y line requires Java 11 a

[jira] [Commented] (KAFKA-14912) Introduce a configuration for remote index cache size, preferably a dynamic config.

2023-08-14 Thread hudeqi (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17754089#comment-17754089 ] hudeqi commented on KAFKA-14912: Hi, [~divijvaidya] Caffeine does support “a max size by

[GitHub] [kafka] clolov commented on pull request #14176: KAFKA-15295: Add config validation when remote storage is enabled on a topic

2023-08-14 Thread via GitHub
clolov commented on PR #14176: URL: https://github.com/apache/kafka/pull/14176#issuecomment-1677252009 Please correct me if I have understood this wrongly, but this pull request proposes propagating a configuration known only to the RemoteLogManagerConfig to the LogConfig via the KafkaConfi

[jira] [Updated] (KAFKA-15342) Considering upgrading to Mockito 5.4.1 or later

2023-08-14 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton updated KAFKA-15342: -- Description: We're currently stuck on Mockito 4.x.y because the 5.x.y line requires Java 11 a

[GitHub] [kafka] C0urante commented on pull request #14186: KAFKA-14682: Report Mockito unused stubbings during Jenkins build

2023-08-14 Thread via GitHub
C0urante commented on PR #14186: URL: https://github.com/apache/kafka/pull/14186#issuecomment-1677247712 Final note: I've filed [KAFKA-15342](https://issues.apache.org/jira/browse/KAFKA-15342) as a reminder to revert this change if we're able to upgrade to a version of Mockito that include

[jira] [Created] (KAFKA-15342) Considering upgrading to Mockito 5.4.1 or later

2023-08-14 Thread Chris Egerton (Jira)
Chris Egerton created KAFKA-15342: - Summary: Considering upgrading to Mockito 5.4.1 or later Key: KAFKA-15342 URL: https://issues.apache.org/jira/browse/KAFKA-15342 Project: Kafka Issue Type:

[GitHub] [kafka] divijvaidya commented on a diff in pull request #14204: KAFKA-14133: Move mocks from KStreamTransformValuesTest, KTableImplTest and KTableTransformValuesTest to Mockito

2023-08-14 Thread via GitHub
divijvaidya commented on code in PR #14204: URL: https://github.com/apache/kafka/pull/14204#discussion_r1293407332 ## streams/src/test/java/org/apache/kafka/streams/kstream/internals/KTableTransformValuesTest.java: ## @@ -163,104 +159,74 @@ public void shouldNotSendOldValuesByDe

[jira] [Resolved] (KAFKA-14682) Unused stubbings are not reported by Mockito during CI builds

2023-08-14 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton resolved KAFKA-14682. --- Fix Version/s: 3.6.0 Resolution: Fixed > Unused stubbings are not reported by Mockito

[GitHub] [kafka] nikramakrishnan commented on a diff in pull request #13561: KAFKA-14888: Added remote log segments retention functionality based on time and size.

2023-08-14 Thread via GitHub
nikramakrishnan commented on code in PR #13561: URL: https://github.com/apache/kafka/pull/13561#discussion_r1293402836 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -698,11 +707,329 @@ public void run() { } } +public void han

[jira] [Updated] (KAFKA-15341) Enabling TS for a topic during rolling restart causes problems

2023-08-14 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya updated KAFKA-15341: - Description: When we are in a rolling restart to enable TS at system level, some brokers have T

[jira] [Updated] (KAFKA-15341) Enabling TS for a topic during rolling restart causes problems

2023-08-14 Thread Divij Vaidya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya updated KAFKA-15341: - Labels: KIP-405 (was: ) > Enabling TS for a topic during rolling restart causes problems >

[GitHub] [kafka] kamalcph commented on pull request #14176: KAFKA-15295: Add config validation when remote storage is enabled on a topic

2023-08-14 Thread via GitHub
kamalcph commented on PR #14176: URL: https://github.com/apache/kafka/pull/14176#issuecomment-1677211747 @divijvaidya @showuon Addressed your review comments. PTAL. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [kafka] kamalcph commented on a diff in pull request #14176: KAFKA-15295: Add config validation when remote storage is enabled on a topic

2023-08-14 Thread via GitHub
kamalcph commented on code in PR #14176: URL: https://github.com/apache/kafka/pull/14176#discussion_r1293382005 ## core/src/test/scala/integration/kafka/admin/RemoteTopicCRUDTest.scala: ## @@ -0,0 +1,262 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or m

[GitHub] [kafka] clolov opened a new pull request, #14205: KAFKA-14133: Migrate SessionCacheFlushListenerTest, TimestampedCacheFlushListenerTest and TimestampedTupleForwarderTest to Mockito

2023-08-14 Thread via GitHub
clolov opened a new pull request, #14205: URL: https://github.com/apache/kafka/pull/14205 This pull requests migrates mocks from FlushListenerTest, TimestampedCacheFlushListenerTest and TimestampedTupleForwarderTest to Mockito. -- This is an automated message from the Apache Git Service.

[GitHub] [kafka] kamalcph commented on a diff in pull request #14176: KAFKA-15295: Add config validation when remote storage is enabled on a topic

2023-08-14 Thread via GitHub
kamalcph commented on code in PR #14176: URL: https://github.com/apache/kafka/pull/14176#discussion_r1293372005 ## core/src/test/scala/integration/kafka/admin/RemoteTopicCRUDTest.scala: ## @@ -0,0 +1,262 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or m

[GitHub] [kafka] kamalcph commented on a diff in pull request #14176: KAFKA-15295: Add config validation when remote storage is enabled on a topic

2023-08-14 Thread via GitHub
kamalcph commented on code in PR #14176: URL: https://github.com/apache/kafka/pull/14176#discussion_r1293370756 ## core/src/test/scala/integration/kafka/admin/RemoteTopicCRUDTest.scala: ## @@ -0,0 +1,262 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or m

[GitHub] [kafka] omkreddy commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-14 Thread via GitHub
omkreddy commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1293243166 ## core/src/main/scala/kafka/server/ControllerApis.scala: ## @@ -842,6 +847,75 @@ class ControllerApis(val requestChannel: RequestChannel, } } + def handl

[GitHub] [kafka] divijvaidya commented on a diff in pull request #14176: KAFKA-15295: Add config validation when remote storage is enabled on a topic

2023-08-14 Thread via GitHub
divijvaidya commented on code in PR #14176: URL: https://github.com/apache/kafka/pull/14176#discussion_r1293349126 ## core/src/main/scala/kafka/server/ControllerServer.scala: ## @@ -231,7 +231,7 @@ class ControllerServer( setMetrics(quorumControllerMetrics).

[GitHub] [kafka] divijvaidya commented on a diff in pull request #14176: KAFKA-15295: Add config validation when remote storage is enabled on a topic

2023-08-14 Thread via GitHub
divijvaidya commented on code in PR #14176: URL: https://github.com/apache/kafka/pull/14176#discussion_r1293348162 ## core/src/main/scala/kafka/server/ControllerServer.scala: ## @@ -231,7 +231,7 @@ class ControllerServer( setMetrics(quorumControllerMetrics).

[GitHub] [kafka] satishd commented on a diff in pull request #13561: KAFKA-14888: Added remote log segments retention functionality based on time and size.

2023-08-14 Thread via GitHub
satishd commented on code in PR #13561: URL: https://github.com/apache/kafka/pull/13561#discussion_r1293345266 ## core/src/main/scala/kafka/log/UnifiedLog.scala: ## @@ -1390,7 +1424,15 @@ class UnifiedLog(@volatile var logStartOffset: Long, private def deleteOldSegments(predi

[GitHub] [kafka] satishd commented on a diff in pull request #13561: KAFKA-14888: Added remote log segments retention functionality based on time and size.

2023-08-14 Thread via GitHub
satishd commented on code in PR #13561: URL: https://github.com/apache/kafka/pull/13561#discussion_r1293344902 ## core/src/main/scala/kafka/log/UnifiedLog.scala: ## @@ -1390,7 +1424,15 @@ class UnifiedLog(@volatile var logStartOffset: Long, private def deleteOldSegments(predi

[GitHub] [kafka] divijvaidya commented on a diff in pull request #13561: KAFKA-14888: Added remote log segments retention functionality based on time and size.

2023-08-14 Thread via GitHub
divijvaidya commented on code in PR #13561: URL: https://github.com/apache/kafka/pull/13561#discussion_r1293336126 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -595,6 +609,193 @@ public void run() { } } +public void handleLo

[GitHub] [kafka] divijvaidya commented on a diff in pull request #13561: KAFKA-14888: Added remote log segments retention functionality based on time and size.

2023-08-14 Thread via GitHub
divijvaidya commented on code in PR #13561: URL: https://github.com/apache/kafka/pull/13561#discussion_r1293244517 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -698,11 +707,329 @@ public void run() { } } +public void handleL

[jira] [Updated] (KAFKA-14133) Remaining EasyMock to Mockito tests

2023-08-14 Thread Christo Lolov (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christo Lolov updated KAFKA-14133: -- Description: {color:#de350b}There are tests which use both PowerMock and EasyMock. I have put

[jira] [Updated] (KAFKA-14133) Remaining EasyMock to Mockito tests

2023-08-14 Thread Christo Lolov (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christo Lolov updated KAFKA-14133: -- Description: {color:#de350b}There are tests which use both PowerMock and EasyMock. I have put

[jira] [Resolved] (KAFKA-13836) Improve KRaft broker heartbeat logic

2023-08-14 Thread Deng Ziming (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-13836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deng Ziming resolved KAFKA-13836. - Resolution: Won't Fix > Improve KRaft broker heartbeat logic > -

[GitHub] [kafka] dengziming closed pull request #11951: KAFKA-13836: Improve KRaft broker heartbeat logic

2023-08-14 Thread via GitHub
dengziming closed pull request #11951: KAFKA-13836: Improve KRaft broker heartbeat logic URL: https://github.com/apache/kafka/pull/11951 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [kafka] dengziming commented on pull request #11951: KAFKA-13836: Improve KRaft broker heartbeat logic

2023-08-14 Thread via GitHub
dengziming commented on PR #11951: URL: https://github.com/apache/kafka/pull/11951#issuecomment-1677113212 This has been fixed in #12983, we use `sharedServer.loader.lastAppliedOffset()` which is the published offset. -- This is an automated message from the Apache Git Service. To respond

  1   2   >