[GitHub] [kafka] kamalcph commented on a diff in pull request #13561: KAFKA-14888: Added remote log segments retention functionality based on time and size.

2023-08-15 Thread via GitHub
kamalcph commented on code in PR #13561: URL: https://github.com/apache/kafka/pull/13561#discussion_r1295402786 ## core/src/test/java/kafka/log/remote/RemoteLogManagerTest.java: ## @@ -1003,6 +1015,134 @@ public RemoteLogMetadataManager createRemoteLogMetadataManager() {

[GitHub] [kafka] kamalcph commented on a diff in pull request #13561: KAFKA-14888: Added remote log segments retention functionality based on time and size.

2023-08-15 Thread via GitHub
kamalcph commented on code in PR #13561: URL: https://github.com/apache/kafka/pull/13561#discussion_r1295394486 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -618,6 +625,230 @@ public void run() { } } +public void

[jira] [Updated] (KAFKA-15352) Ensure consistency while deleting the remote log segments

2023-08-15 Thread Kamal Chandraprakash (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamal Chandraprakash updated KAFKA-15352: - Parent: KAFKA-7739 Issue Type: Sub-task (was: Task) > Ensure

[jira] [Created] (KAFKA-15352) Ensure consistency while deleting the remote log segments

2023-08-15 Thread Kamal Chandraprakash (Jira)
Kamal Chandraprakash created KAFKA-15352: Summary: Ensure consistency while deleting the remote log segments Key: KAFKA-15352 URL: https://issues.apache.org/jira/browse/KAFKA-15352 Project:

[jira] [Updated] (KAFKA-15351) Update log-start-offset after leader election for topics enabled with remote storage

2023-08-15 Thread Kamal Chandraprakash (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamal Chandraprakash updated KAFKA-15351: - Description: Case-1: In the FETCH response, the leader-log-start-offset will

[jira] [Created] (KAFKA-15351) Update log-start-offset after leader election for topics enabled with remote storage

2023-08-15 Thread Kamal Chandraprakash (Jira)
Kamal Chandraprakash created KAFKA-15351: Summary: Update log-start-offset after leader election for topics enabled with remote storage Key: KAFKA-15351 URL:

[jira] [Updated] (KAFKA-15351) Update log-start-offset after leader election for topics enabled with remote storage

2023-08-15 Thread Kamal Chandraprakash (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamal Chandraprakash updated KAFKA-15351: - Parent: KAFKA-7739 Issue Type: Sub-task (was: Task) > Update

[GitHub] [kafka] satishd commented on a diff in pull request #13561: KAFKA-14888: Added remote log segments retention functionality based on time and size.

2023-08-15 Thread via GitHub
satishd commented on code in PR #13561: URL: https://github.com/apache/kafka/pull/13561#discussion_r1295387916 ## core/src/main/java/kafka/log/remote/RemoteLogManager.java: ## @@ -1033,6 +1360,35 @@ public void close() { } } +private static class

[jira] [Updated] (KAFKA-15346) Single-Key_single-timestamp IQs with versioned state stores

2023-08-15 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15346: Component/s: streams > Single-Key_single-timestamp IQs with versioned state stores >

[jira] [Updated] (KAFKA-15347) Single-Key_multi-timestamp IQs with versioned state stores

2023-08-15 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15347: Component/s: streams > Single-Key_multi-timestamp IQs with versioned state stores >

[jira] [Updated] (KAFKA-15348) Range IQs with versioned state stores

2023-08-15 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15348: Component/s: streams > Range IQs with versioned state stores >

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-15 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1295339433 ## core/src/main/scala/kafka/server/ControllerApis.scala: ## @@ -842,6 +847,75 @@ class ControllerApis(val requestChannel: RequestChannel, } } + def

[jira] [Resolved] (KAFKA-15329) Make default `remote.log.metadata.manager.class.name` as topic based RLMM

2023-08-15 Thread Luke Chen (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Chen resolved KAFKA-15329. --- Resolution: Fixed > Make default `remote.log.metadata.manager.class.name` as topic based RLMM >

[GitHub] [kafka] showuon merged pull request #14202: KAFKA-15329: Make default remote.log.metadata.manager.class.name as topic based RLMM

2023-08-15 Thread via GitHub
showuon merged PR #14202: URL: https://github.com/apache/kafka/pull/14202 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] showuon commented on pull request #14202: KAFKA-15329: Make default remote.log.metadata.manager.class.name as topic based RLMM

2023-08-15 Thread via GitHub
showuon commented on PR #14202: URL: https://github.com/apache/kafka/pull/14202#issuecomment-1679845654 Failed tests are unrelated: ``` Build / JDK 11 and Scala 2.13 / kafka.server.DynamicBrokerReconfigurationTest.testThreadPoolResize() Build / JDK 11 and Scala 2.13 /

[GitHub] [kafka] pprovenzano commented on pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-15 Thread via GitHub
pprovenzano commented on PR #14083: URL: https://github.com/apache/kafka/pull/14083#issuecomment-1679832173 > I think this starts to look good. There are some parts I haven't reviewed yet, I'll try to get them in the next few days. Thank you for the review comments. I think I have

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-15 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1295289182 ## core/src/main/scala/kafka/server/DelegationTokenManager.scala: ## @@ -186,57 +134,28 @@ class DelegationTokenManager(val config: KafkaConfig, val

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-15 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1295288501 ## core/src/main/scala/kafka/server/metadata/DelegationTokenPublisher.scala: ## @@ -0,0 +1,83 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] [kafka] pprovenzano commented on pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-15 Thread via GitHub
pprovenzano commented on PR #14083: URL: https://github.com/apache/kafka/pull/14083#issuecomment-1679827453 > @pprovenzano Thanks for the PR. left few comments > > can we also update delegation token docs if required (like any configs for controller nodes etc..)

[GitHub] [kafka] gharris1727 commented on a diff in pull request #14220: KAFKA-15102: Add release notes about the replication.policy.internal.topic.separator.enabled property for MirrorMaker 2

2023-08-15 Thread via GitHub
gharris1727 commented on code in PR #14220: URL: https://github.com/apache/kafka/pull/14220#discussion_r1295265105 ## docs/upgrade.html: ## @@ -43,6 +43,10 @@ Notable changes in 3 See

[GitHub] [kafka] C0urante commented on a diff in pull request #14220: KAFKA-15102: Add release notes about the replication.policy.internal.topic.separator.enabled property for MirrorMaker 2

2023-08-15 Thread via GitHub
C0urante commented on code in PR #14220: URL: https://github.com/apache/kafka/pull/14220#discussion_r1295254068 ## docs/upgrade.html: ## @@ -43,6 +43,10 @@ Notable changes in 3 See

[GitHub] [kafka] C0urante commented on pull request #14220: KAFKA-15102: Add release notes about the replication.policy.internal.topic.separator.enabled property for MirrorMaker 2

2023-08-15 Thread via GitHub
C0urante commented on PR #14220: URL: https://github.com/apache/kafka/pull/14220#issuecomment-1679781965 @gharris1727 @mimaison as the other two committers who voted for KIP-949, would you mind taking a look? Thanks! -- This is an automated message from the Apache Git Service. To respond

[GitHub] [kafka] C0urante opened a new pull request, #14220: KAFKA-15102: Add release notes about the replication.policy.internal.topic.separator.enabled property for MirrorMaker 2

2023-08-15 Thread via GitHub
C0urante opened a new pull request, #14220: URL: https://github.com/apache/kafka/pull/14220 [Jira](https://issues.apache.org/jira/browse/KAFKA-15102) This is a docs-only follow-up for https://github.com/apache/kafka/pull/14082 that calls out the necessity for this property in our

[GitHub] [kafka] C0urante merged pull request #14082: KAFKA-15102: Mirror Maker 2 - KIP690 backward compatibility

2023-08-15 Thread via GitHub
C0urante merged PR #14082: URL: https://github.com/apache/kafka/pull/14082 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] mumrah commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-15 Thread via GitHub
mumrah commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1295237042 ## metadata/src/main/java/org/apache/kafka/image/loader/LogDeltaManifest.java: ## @@ -66,6 +119,10 @@ public LogDeltaManifest( this.numBytes = numBytes; }

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-15 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1295230735 ## metadata/src/test/java/org/apache/kafka/image/DelegationTokenImageTest.java: ## Review Comment: Yes and Done -- This is an automated message from the

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-15 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1295230635 ## metadata/src/test/java/org/apache/kafka/image/MetadataImageTest.java: ## Review Comment: Fixed -- This is an automated message from the Apache Git

[jira] [Resolved] (KAFKA-15350) MetadataLoaderMetrics has ClassNotFoundException in system tests

2023-08-15 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Harris resolved KAFKA-15350. - Resolution: Invalid I believe this was an environmental problem, due to some stale artifacts. A

[jira] [Updated] (KAFKA-15350) MetadataLoaderMetrics has ClassNotFoundException in system tests

2023-08-15 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Harris updated KAFKA-15350: Issue Type: Bug (was: Improvement) > MetadataLoaderMetrics has ClassNotFoundException in system

[jira] [Updated] (KAFKA-15336) Connect plugin Javadocs should mention serviceloader manifests

2023-08-15 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Harris updated KAFKA-15336: Issue Type: Improvement (was: Bug) > Connect plugin Javadocs should mention serviceloader

[jira] [Updated] (KAFKA-15336) Connect plugin Javadocs should mention serviceloader manifests

2023-08-15 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Harris updated KAFKA-15336: Issue Type: Bug (was: Improvement) > Connect plugin Javadocs should mention serviceloader

[jira] [Created] (KAFKA-15350) MetadataLoaderMetrics has ClassNotFoundException in system tests

2023-08-15 Thread Greg Harris (Jira)
Greg Harris created KAFKA-15350: --- Summary: MetadataLoaderMetrics has ClassNotFoundException in system tests Key: KAFKA-15350 URL: https://issues.apache.org/jira/browse/KAFKA-15350 Project: Kafka

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #14182: KAFKA 14515: Optimized Uniform Rack Aware Assignor

2023-08-15 Thread via GitHub
rreddy-22 commented on code in PR #14182: URL: https://github.com/apache/kafka/pull/14182#discussion_r1295198962 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/OptimizedUniformAssignmentBuilder.java: ## @@ -0,0 +1,399 @@ +/* + * Licensed to the

[GitHub] [kafka] rreddy-22 commented on a diff in pull request #14182: KAFKA 14515: Optimized Uniform Rack Aware Assignor

2023-08-15 Thread via GitHub
rreddy-22 commented on code in PR #14182: URL: https://github.com/apache/kafka/pull/14182#discussion_r1295197016 ## group-coordinator/src/main/java/org/apache/kafka/coordinator/group/assignor/OptimizedUniformAssignmentBuilder.java: ## @@ -0,0 +1,399 @@ +/* + * Licensed to the

[GitHub] [kafka] lucasbru commented on pull request #14216: KAFKA-15319: Upgrade rocksdb to fix CVE-2022-37434

2023-08-15 Thread via GitHub
lucasbru commented on PR #14216: URL: https://github.com/apache/kafka/pull/14216#issuecomment-1679690414 https://jenkins.confluent.io/job/confluentinc/job/kafka-streams-benchmarks/job/master/653/parameters/

[GitHub] [kafka] junrao merged pull request #14179: MINOR: CommitRequestManager should only poll when the coordinator node is known

2023-08-15 Thread via GitHub
junrao merged PR #14179: URL: https://github.com/apache/kafka/pull/14179 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[jira] [Created] (KAFKA-15349) ducker-ak should fail fast when gradlew systemTestLibs fails

2023-08-15 Thread Greg Harris (Jira)
Greg Harris created KAFKA-15349: --- Summary: ducker-ak should fail fast when gradlew systemTestLibs fails Key: KAFKA-15349 URL: https://issues.apache.org/jira/browse/KAFKA-15349 Project: Kafka

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-15 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1295140057 ## metadata/src/test/java/org/apache/kafka/image/MetadataImageTest.java: ## Review Comment: It is because the compare is using the toString() which redacts

[GitHub] [kafka] gharris1727 merged pull request #14177: MINOR: Fix SynchronizationTest Classloaders sometimes not being parallel capable

2023-08-15 Thread via GitHub
gharris1727 merged PR #14177: URL: https://github.com/apache/kafka/pull/14177 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] cmccabe commented on pull request #14213: KAFKA-15345; KRaft leader notifies when listener reaches epoch start

2023-08-15 Thread via GitHub
cmccabe commented on PR #14213: URL: https://github.com/apache/kafka/pull/14213#issuecomment-1679620704 > The RecordsBatchReader implementation is also changed to include control records. This makes it possible for the state machine learn about committed control records. This additional

[GitHub] [kafka] gharris1727 commented on pull request #14177: MINOR: Fix SynchronizationTest Classloaders sometimes not being parallel capable

2023-08-15 Thread via GitHub
gharris1727 commented on PR #14177: URL: https://github.com/apache/kafka/pull/14177#issuecomment-1679635472 CI failures appear unrelated, and the tests pass locally. The test being fixed shows no failures. -- This is an automated message from the Apache Git Service. To respond to the

[jira] [Commented] (KAFKA-15343) Fix MirrorConnectIntegrationTests causing ci build failures.

2023-08-15 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17754771#comment-17754771 ] Greg Harris commented on KAFKA-15343: - Hi [~prasanth] and thank you for reporting this issue! It is

[GitHub] [kafka] jeqo opened a new pull request, #14219: KIP-405: 2023-08-15

2023-08-15 Thread via GitHub
jeqo opened a new pull request, #14219: URL: https://github.com/apache/kafka/pull/14219 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of testing

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-15 Thread via GitHub
cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1295077023 ## metadata/src/main/java/org/apache/kafka/image/loader/LogDeltaManifest.java: ## @@ -80,6 +137,7 @@ public LeaderAndEpoch leaderAndEpoch() { return

[GitHub] [kafka] cmccabe commented on pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-15 Thread via GitHub
cmccabe commented on PR #14208: URL: https://github.com/apache/kafka/pull/14208#issuecomment-1679586281 Thanks for the PR, @mumrah . I think at some point we'll need to add begin/end transactions to the set of things we "fuzz". Basically have a test that just makes sure that the

[GitHub] [kafka] jeqo closed pull request #14219: KIP-405: 2023-08-15

2023-08-15 Thread via GitHub
jeqo closed pull request #14219: KIP-405: 2023-08-15 URL: https://github.com/apache/kafka/pull/14219 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-15 Thread via GitHub
cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1295076740 ## metadata/src/main/java/org/apache/kafka/image/loader/LogDeltaManifest.java: ## @@ -27,6 +27,59 @@ * Contains information about a set of changes that were loaded

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-15 Thread via GitHub
cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1295091800 ## metadata/src/main/java/org/apache/kafka/image/loader/MetadataBatchLoader.java: ## @@ -0,0 +1,257 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] [kafka] lianetm commented on a diff in pull request #14218: KAFKA-14937; [2/N]: Refactoring for client code to reduce boilerplate

2023-08-15 Thread via GitHub
lianetm commented on code in PR #14218: URL: https://github.com/apache/kafka/pull/14218#discussion_r1295078074 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerUtils.java: ## @@ -134,44 +132,20 @@ public static FetchMetricsManager

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-15 Thread via GitHub
cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1295084331 ## metadata/src/main/java/org/apache/kafka/image/loader/MetadataBatchLoader.java: ## @@ -0,0 +1,257 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] [kafka] gharris1727 commented on pull request #14194: KAFKA-15336: Add ServiceLoader Javadocs for Connect plugins

2023-08-15 Thread via GitHub
gharris1727 commented on PR #14194: URL: https://github.com/apache/kafka/pull/14194#issuecomment-1679551865 This is a documentation-only change, and test failures appear unrelated. A local `javadoc` build doesn't include any errors for these classes. -- This is an automated message from

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-15 Thread via GitHub
cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1295090413 ## metadata/src/main/java/org/apache/kafka/image/loader/MetadataLoader.java: ## @@ -183,6 +183,8 @@ public MetadataLoader build() { */ private MetadataImage

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-15 Thread via GitHub
cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1295084521 ## metadata/src/main/java/org/apache/kafka/image/loader/MetadataBatchLoader.java: ## @@ -0,0 +1,257 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-15 Thread via GitHub
cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1295077023 ## metadata/src/main/java/org/apache/kafka/image/loader/LogDeltaManifest.java: ## @@ -80,6 +137,7 @@ public LeaderAndEpoch leaderAndEpoch() { return

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-15 Thread via GitHub
cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1295081634 ## metadata/src/main/java/org/apache/kafka/image/loader/MetadataBatchLoader.java: ## @@ -0,0 +1,257 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-15 Thread via GitHub
cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1295077448 ## metadata/src/main/java/org/apache/kafka/image/loader/LogDeltaManifest.java: ## @@ -66,6 +119,10 @@ public LogDeltaManifest( this.numBytes = numBytes;

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-15 Thread via GitHub
cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1295074841 ## metadata/src/main/java/org/apache/kafka/controller/QuorumController.java: ## @@ -1200,6 +1235,16 @@ public static List generateActivationRecords(

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-15 Thread via GitHub
cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1295072484 ## metadata/src/main/java/org/apache/kafka/controller/QuorumController.java: ## @@ -892,48 +909,62 @@ class MigrationWriteOperation implements

[GitHub] [kafka] gharris1727 merged pull request #14194: KAFKA-15336: Add ServiceLoader Javadocs for Connect plugins

2023-08-15 Thread via GitHub
gharris1727 merged PR #14194: URL: https://github.com/apache/kafka/pull/14194 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-15 Thread via GitHub
cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1295073210 ## metadata/src/main/java/org/apache/kafka/controller/QuorumController.java: ## @@ -954,11 +985,14 @@ public void handleCommit(BatchReader reader) {

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-15 Thread via GitHub
cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1295069992 ## metadata/src/main/java/org/apache/kafka/controller/QuorumController.java: ## @@ -623,7 +624,14 @@ enum ControllerOperationFlag { * even though the cluster

[GitHub] [kafka] cmccabe commented on a diff in pull request #14208: KAFKA-14538 Metadata transactions in MetadataLoader

2023-08-15 Thread via GitHub
cmccabe commented on code in PR #14208: URL: https://github.com/apache/kafka/pull/14208#discussion_r1295071752 ## metadata/src/main/java/org/apache/kafka/controller/QuorumController.java: ## @@ -892,48 +909,62 @@ class MigrationWriteOperation implements

[GitHub] [kafka] lianetm commented on a diff in pull request #14218: KAFKA-14937; [2/N]: Refactoring for client code to reduce boilerplate

2023-08-15 Thread via GitHub
lianetm commented on code in PR #14218: URL: https://github.com/apache/kafka/pull/14218#discussion_r1295049009 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerUtils.java: ## @@ -134,44 +132,20 @@ public static FetchMetricsManager

[GitHub] [kafka] lianetm commented on a diff in pull request #14218: KAFKA-14937; [2/N]: Refactoring for client code to reduce boilerplate

2023-08-15 Thread via GitHub
lianetm commented on code in PR #14218: URL: https://github.com/apache/kafka/pull/14218#discussion_r1295036974 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerUtils.java: ## @@ -134,44 +132,20 @@ public static FetchMetricsManager

[GitHub] [kafka] AndrewJSchofield commented on a diff in pull request #14111: KAFKA-9800: Exponential backoff for Kafka clients - KIP-580

2023-08-15 Thread via GitHub
AndrewJSchofield commented on code in PR #14111: URL: https://github.com/apache/kafka/pull/14111#discussion_r1295024473 ## clients/src/main/java/org/apache/kafka/clients/Metadata.java: ## @@ -114,18 +125,39 @@ public synchronized Cluster fetch() { /** * Return the

[GitHub] [kafka] cmccabe merged pull request #14215: MINOR Install metadata publishers sooner in ControllerServer

2023-08-15 Thread via GitHub
cmccabe merged PR #14215: URL: https://github.com/apache/kafka/pull/14215 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] cmccabe closed pull request #14199: MINOR Fix the ZkMigrationState metric in KafkaController

2023-08-15 Thread via GitHub
cmccabe closed pull request #14199: MINOR Fix the ZkMigrationState metric in KafkaController URL: https://github.com/apache/kafka/pull/14199 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [kafka] AndrewJSchofield commented on a diff in pull request #14111: KAFKA-9800: Exponential backoff for Kafka clients - KIP-580

2023-08-15 Thread via GitHub
AndrewJSchofield commented on code in PR #14111: URL: https://github.com/apache/kafka/pull/14111#discussion_r1295030179 ## clients/src/main/java/org/apache/kafka/clients/Metadata.java: ## @@ -140,17 +172,34 @@ public long metadataExpireMs() { } /** - * Request

[GitHub] [kafka] cmccabe commented on pull request #14199: MINOR Fix the ZkMigrationState metric in KafkaController

2023-08-15 Thread via GitHub
cmccabe commented on PR #14199: URL: https://github.com/apache/kafka/pull/14199#issuecomment-1679483573 LGTM, committed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [kafka] ruslankrivoshein commented on pull request #13562: KAFKA-14581: Moving GetOffsetShell to tools

2023-08-15 Thread via GitHub
ruslankrivoshein commented on PR #13562: URL: https://github.com/apache/kafka/pull/13562#issuecomment-1679474019 @mimaison please, take a look -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [kafka] divijvaidya commented on a diff in pull request #14161: KAFKA-15267: Do not allow Tiered Storage to be disabled while topics have remote.storage.enable property

2023-08-15 Thread via GitHub
divijvaidya commented on code in PR #14161: URL: https://github.com/apache/kafka/pull/14161#discussion_r1294991302 ## core/src/test/scala/unit/kafka/server/KafkaServerTest.scala: ## @@ -154,6 +155,96 @@ class KafkaServerTest extends QuorumTestHarness { server.shutdown()

[GitHub] [kafka] divijvaidya commented on a diff in pull request #14161: KAFKA-15267: Do not allow Tiered Storage to be disabled while topics have remote.storage.enable property

2023-08-15 Thread via GitHub
divijvaidya commented on code in PR #14161: URL: https://github.com/apache/kafka/pull/14161#discussion_r1294983667 ## core/src/test/scala/unit/kafka/server/KafkaServerTest.scala: ## @@ -154,6 +155,96 @@ class KafkaServerTest extends QuorumTestHarness { server.shutdown()

[GitHub] [kafka] divijvaidya commented on a diff in pull request #14161: KAFKA-15267: Do not allow Tiered Storage to be disabled while topics have remote.storage.enable property

2023-08-15 Thread via GitHub
divijvaidya commented on code in PR #14161: URL: https://github.com/apache/kafka/pull/14161#discussion_r1294994048 ## core/src/main/scala/kafka/server/ConfigHandler.scala: ## @@ -62,6 +62,12 @@ class TopicConfigHandler(private val logManager: LogManager, kafkaConfig: KafkaC

[GitHub] [kafka] philipnee commented on a diff in pull request #14218: KAFKA-14937; [2/N]: Refactoring for client code to reduce boilerplate

2023-08-15 Thread via GitHub
philipnee commented on code in PR #14218: URL: https://github.com/apache/kafka/pull/14218#discussion_r1294990279 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/ConsumerUtils.java: ## @@ -134,44 +132,20 @@ public static FetchMetricsManager

[GitHub] [kafka] divijvaidya commented on pull request #14176: KAFKA-15295: Add config validation when remote storage is enabled on a topic

2023-08-15 Thread via GitHub
divijvaidya commented on PR #14176: URL: https://github.com/apache/kafka/pull/14176#issuecomment-1679415679 Unrelated test failures: ``` [Build / JDK 17 and Scala 2.13 /

[GitHub] [kafka] divijvaidya merged pull request #14176: KAFKA-15295: Add config validation when remote storage is enabled on a topic

2023-08-15 Thread via GitHub
divijvaidya merged PR #14176: URL: https://github.com/apache/kafka/pull/14176 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] divijvaidya commented on pull request #14212: MINOR: Add test case for follower fetch

2023-08-15 Thread via GitHub
divijvaidya commented on PR #14212: URL: https://github.com/apache/kafka/pull/14212#issuecomment-1679398284 Thank you for adding this. Please add what you wrote in the description as a comment in the test so that reader and quickly understand what the test is supposed to check.

[GitHub] [kafka] divijvaidya commented on a diff in pull request #14212: MINOR: Add test case for follower fetch

2023-08-15 Thread via GitHub
divijvaidya commented on code in PR #14212: URL: https://github.com/apache/kafka/pull/14212#discussion_r1294964226 ## core/src/test/scala/unit/kafka/server/ReplicaFetcherTierStateMachineTest.scala: ## @@ -85,6 +85,64 @@ class ReplicaFetcherTierStateMachineTest {

[GitHub] [kafka] lianetm opened a new pull request, #14218: KAFKA-14937; [2/N]: Refactoring for client code to reduce boilerplate

2023-08-15 Thread via GitHub
lianetm opened a new pull request, #14218: URL: https://github.com/apache/kafka/pull/14218 This is a continuation of the previous [PR#13990](https://github.com/apache/kafka/pull/13990) This PR main refactoring relates to : - serializers/deserializers used in clients - unified in a

[GitHub] [kafka] nizhikov opened a new pull request, #14217: KAFKA-14595 ReassignPartitionsCommandArgsTest rewritten in java

2023-08-15 Thread via GitHub
nizhikov opened a new pull request, #14217: URL: https://github.com/apache/kafka/pull/14217 This PR is part of #13247 It includes `ReassignPartitionsCommandArgsTest` rewritten in java. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation

[GitHub] [kafka] nizhikov commented on pull request #13247: KAFKA-14595 Move ReassignPartitionsCommand to java

2023-08-15 Thread via GitHub
nizhikov commented on PR #13247: URL: https://github.com/apache/kafka/pull/13247#issuecomment-1679370921 @mimaison I create #14217 which include `ReassignPartitionsCommandArgsTest` rewritten in java. It independent from the big PR and can be reviewed separately. Can you, please, take a

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-15 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1294925543 ## metadata/src/main/resources/common/metadata/DelegationTokenRecord.json: ## @@ -22,8 +22,10 @@ "fields": [ { "name": "Owner", "type": "string",

[GitHub] [kafka] pprovenzano commented on a diff in pull request #14083: KAFKA-15219: KRaft support for DelegationTokens

2023-08-15 Thread via GitHub
pprovenzano commented on code in PR #14083: URL: https://github.com/apache/kafka/pull/14083#discussion_r1294918864 ## server-common/src/main/java/org/apache/kafka/server/common/MetadataVersion.java: ## @@ -267,6 +270,10 @@ public boolean isLeaderEpochBumpRequiredOnIsrShrink() {

[jira] [Updated] (KAFKA-14133) Remaining EasyMock to Mockito tests

2023-08-15 Thread Yash Mayya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yash Mayya updated KAFKA-14133: --- Description: {color:#de350b}There are tests which use both PowerMock and EasyMock. I have put

[jira] [Updated] (KAFKA-14133) Remaining EasyMock to Mockito tests

2023-08-15 Thread Yash Mayya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yash Mayya updated KAFKA-14133: --- Description: {color:#de350b}There are tests which use both PowerMock and EasyMock. I have put

[jira] [Updated] (KAFKA-15344) Kafka Streams should include the message leader epoch when committing offsets

2023-08-15 Thread David Mao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mao updated KAFKA-15344: -- Description: We noticed an application received an OFFSET_OUT_OF_RANGE error following a network

[GitHub] [kafka] AndrewJSchofield commented on pull request #14111: KAFKA-9800: Exponential backoff for Kafka clients - KIP-580

2023-08-15 Thread via GitHub
AndrewJSchofield commented on PR #14111: URL: https://github.com/apache/kafka/pull/14111#issuecomment-1679338770 @junrao I don't know what's causing so many test failures. 78 of them were due to "unexpected threads" in the broker. I'll take another look when there's another build. I expect

[GitHub] [kafka] junrao commented on a diff in pull request #14111: KAFKA-9800: Exponential backoff for Kafka clients - KIP-580

2023-08-15 Thread via GitHub
junrao commented on code in PR #14111: URL: https://github.com/apache/kafka/pull/14111#discussion_r1294877936 ## clients/src/main/java/org/apache/kafka/clients/Metadata.java: ## @@ -267,11 +307,14 @@ public synchronized void update(int requestVersion, MetadataResponse

[GitHub] [kafka] junrao commented on a diff in pull request #14111: KAFKA-9800: Exponential backoff for Kafka clients - KIP-580

2023-08-15 Thread via GitHub
junrao commented on code in PR #14111: URL: https://github.com/apache/kafka/pull/14111#discussion_r1294883198 ## clients/src/main/java/org/apache/kafka/clients/Metadata.java: ## @@ -140,17 +172,34 @@ public long metadataExpireMs() { } /** - * Request an update

[GitHub] [kafka] junrao commented on a diff in pull request #14111: KAFKA-9800: Exponential backoff for Kafka clients - KIP-580

2023-08-15 Thread via GitHub
junrao commented on code in PR #14111: URL: https://github.com/apache/kafka/pull/14111#discussion_r1294873505 ## clients/src/main/java/org/apache/kafka/clients/Metadata.java: ## @@ -114,18 +125,39 @@ public synchronized Cluster fetch() { /** * Return the next time

[jira] [Updated] (KAFKA-14133) Remaining EasyMock to Mockito tests

2023-08-15 Thread Yash Mayya (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-14133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yash Mayya updated KAFKA-14133: --- Description: {color:#de350b}There are tests which use both PowerMock and EasyMock. I have put

[jira] [Updated] (KAFKA-15344) Kafka Streams should include the message leader epoch when committing offsets

2023-08-15 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15344: Component/s: streams > Kafka Streams should include the message leader epoch when

[GitHub] [kafka] divijvaidya commented on a diff in pull request #14176: KAFKA-15295: Add config validation when remote storage is enabled on a topic

2023-08-15 Thread via GitHub
divijvaidya commented on code in PR #14176: URL: https://github.com/apache/kafka/pull/14176#discussion_r1294874991 ## core/src/test/scala/integration/kafka/admin/RemoteTopicCrudTest.scala: ## @@ -0,0 +1,331 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one

[jira] [Updated] (KAFKA-15257) Support interactive queries (IQv2) with versioned state store

2023-08-15 Thread Alieh Saeedi (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alieh Saeedi updated KAFKA-15257: - Description: Query types to consider include: * single-key latest-value lookup * single-key

[jira] [Assigned] (KAFKA-15347) Single-Key_multi-timestamp IQs with versioned state stores

2023-08-15 Thread Alieh Saeedi (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alieh Saeedi reassigned KAFKA-15347: Assignee: Alieh Saeedi > Single-Key_multi-timestamp IQs with versioned state stores >

[jira] [Created] (KAFKA-15348) Range IQs with versioned state stores

2023-08-15 Thread Alieh Saeedi (Jira)
Alieh Saeedi created KAFKA-15348: Summary: Range IQs with versioned state stores Key: KAFKA-15348 URL: https://issues.apache.org/jira/browse/KAFKA-15348 Project: Kafka Issue Type: Sub-task

[jira] [Assigned] (KAFKA-15348) Range IQs with versioned state stores

2023-08-15 Thread Alieh Saeedi (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alieh Saeedi reassigned KAFKA-15348: Assignee: Alieh Saeedi > Range IQs with versioned state stores >

[jira] [Created] (KAFKA-15347) Single-Key_multi-timestamp IQs with versioned state stores

2023-08-15 Thread Alieh Saeedi (Jira)
Alieh Saeedi created KAFKA-15347: Summary: Single-Key_multi-timestamp IQs with versioned state stores Key: KAFKA-15347 URL: https://issues.apache.org/jira/browse/KAFKA-15347 Project: Kafka

[jira] [Updated] (KAFKA-15346) Single-Key_single-timestamp IQs with versioned state stores

2023-08-15 Thread Alieh Saeedi (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alieh Saeedi updated KAFKA-15346: - Summary: Single-Key_single-timestamp IQs with versioned state stores (was:

[jira] [Updated] (KAFKA-15346) Single-Key_single-multi-timestamp IQs with versioned state stores

2023-08-15 Thread Alieh Saeedi (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alieh Saeedi updated KAFKA-15346: - Description:

  1   2   >