[GitHub] [kafka] satishd commented on a diff in pull request #14382: KAFKA-15442: add a section in doc for tiered storage

2023-09-13 Thread via GitHub
satishd commented on code in PR #14382: URL: https://github.com/apache/kafka/pull/14382#discussion_r1325382836 ## docs/ops.html: ## @@ -3859,6 +3859,98 @@ Finalizing the migration # Other configs ... + +6.11 Tiered Storage + +Tiered Storage Overview + +Kafka data is

[GitHub] [kafka] satishd commented on a diff in pull request #14382: KAFKA-15442: add a section in doc for tiered storage

2023-09-13 Thread via GitHub
satishd commented on code in PR #14382: URL: https://github.com/apache/kafka/pull/14382#discussion_r1325382836 ## docs/ops.html: ## @@ -3859,6 +3859,98 @@ Finalizing the migration # Other configs ... + +6.11 Tiered Storage + +Tiered Storage Overview + +Kafka data is

[GitHub] [kafka] philipnee commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests

2023-09-13 Thread via GitHub
philipnee commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1325383326 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -0,0 +1,260 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] philipnee commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests

2023-09-13 Thread via GitHub
philipnee commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1325382420 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -0,0 +1,260 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] philipnee commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests

2023-09-13 Thread via GitHub
philipnee commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1325381139 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java: ## @@ -0,0 +1,260 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] showuon commented on pull request #14382: KAFKA-15442: add a section in doc for tiered storage

2023-09-13 Thread via GitHub
showuon commented on PR #14382: URL: https://github.com/apache/kafka/pull/14382#issuecomment-1718792709 @satishd , in the latest commit, I added: 1. a note to say this is an early access feature and should not use in production env. 2. a limitation section to describe the limitation

[GitHub] [kafka] philipnee commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests

2023-09-13 Thread via GitHub
philipnee commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1325368170 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/DefaultBackgroundThread.java: ## @@ -168,28 +171,41 @@ public DefaultBackgroundThread(final Time

[GitHub] [kafka] philipnee commented on pull request #14386: KAFKA-14960: TopicMetadata request manager

2023-09-13 Thread via GitHub
philipnee commented on PR #14386: URL: https://github.com/apache/kafka/pull/14386#issuecomment-1718779561 Hey @junrao - Addressed your comments. I'm holding off on your request for `Could we change toString to include the fields in the parent class?` as I think it would be easier if I do

[GitHub] [kafka] philipnee commented on a diff in pull request #14386: KAFKA-14960: TopicMetadata request manager

2023-09-13 Thread via GitHub
philipnee commented on code in PR #14386: URL: https://github.com/apache/kafka/pull/14386#discussion_r1325364414 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/TopicMetadataRequestManager.java: ## @@ -0,0 +1,210 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] philipnee commented on a diff in pull request #14386: KAFKA-14960: TopicMetadata request manager

2023-09-13 Thread via GitHub
philipnee commented on code in PR #14386: URL: https://github.com/apache/kafka/pull/14386#discussion_r1325355774 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/TopicMetadataRequestManager.java: ## @@ -0,0 +1,210 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] philipnee commented on a diff in pull request #14386: KAFKA-14960: TopicMetadata request manager

2023-09-13 Thread via GitHub
philipnee commented on code in PR #14386: URL: https://github.com/apache/kafka/pull/14386#discussion_r1325353270 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/NetworkClientDelegate.java: ## @@ -203,16 +204,17 @@ public static class UnsentRequest {

[GitHub] [kafka] philipnee commented on a diff in pull request #14386: KAFKA-14960: TopicMetadata request manager

2023-09-13 Thread via GitHub
philipnee commented on code in PR #14386: URL: https://github.com/apache/kafka/pull/14386#discussion_r1325352296 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -381,41 +377,41 @@ public String toString() { } /**

[GitHub] [kafka] philipnee commented on a diff in pull request #14386: KAFKA-14960: TopicMetadata request manager

2023-09-13 Thread via GitHub
philipnee commented on code in PR #14386: URL: https://github.com/apache/kafka/pull/14386#discussion_r1325349813 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -106,6 +104,7 @@ public NetworkClientDelegate.PollResult

[GitHub] [kafka] philipnee commented on a diff in pull request #14386: KAFKA-14960: TopicMetadata request manager

2023-09-13 Thread via GitHub
philipnee commented on code in PR #14386: URL: https://github.com/apache/kafka/pull/14386#discussion_r1325349677 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/events/CommitApplicationEvent.java: ## @@ -19,22 +19,23 @@ import

[GitHub] [kafka] philipnee commented on a diff in pull request #14386: KAFKA-14960: TopicMetadata request manager

2023-09-13 Thread via GitHub
philipnee commented on code in PR #14386: URL: https://github.com/apache/kafka/pull/14386#discussion_r1325345190 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/TopicMetadataRequestManager.java: ## @@ -0,0 +1,210 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] philipnee commented on a diff in pull request #14386: KAFKA-14960: TopicMetadata request manager

2023-09-13 Thread via GitHub
philipnee commented on code in PR #14386: URL: https://github.com/apache/kafka/pull/14386#discussion_r1325342579 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/TopicMetadataRequestManager.java: ## @@ -0,0 +1,210 @@ +/* + * Licensed to the Apache Software

[GitHub] [kafka] philipnee commented on a diff in pull request #14386: KAFKA-14960: TopicMetadata request manager

2023-09-13 Thread via GitHub
philipnee commented on code in PR #14386: URL: https://github.com/apache/kafka/pull/14386#discussion_r1325341358 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/TopicMetadataRequestManager.java: ## @@ -0,0 +1,210 @@ +/* + * Licensed to the Apache Software

[jira] [Updated] (KAFKA-15245) Improve Tiered Storage Metrics

2023-09-13 Thread Kamal Chandraprakash (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamal Chandraprakash updated KAFKA-15245: - Fix Version/s: 3.6.0 (was: 3.7.0) > Improve Tiered

[jira] [Updated] (KAFKA-15245) Improve Tiered Storage Metrics

2023-09-13 Thread Kamal Chandraprakash (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamal Chandraprakash updated KAFKA-15245: - Description: Rename existing tiered storage metrics to remove ambiguity (was:

[jira] [Commented] (KAFKA-15245) Improve Tiered Storage Metrics

2023-09-13 Thread Kamal Chandraprakash (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764959#comment-17764959 ] Kamal Chandraprakash commented on KAFKA-15245: -- KAFKA-15094 ticket is already filed to

[jira] [Updated] (KAFKA-15453) Enable `testFencingOnTransactionExpiration` in TransactionsWithTieredStoreTest

2023-09-13 Thread Kamal Chandraprakash (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamal Chandraprakash updated KAFKA-15453: - Fix Version/s: 3.6.0 (was: 3.7.0) > Enable

[jira] [Comment Edited] (KAFKA-15160) Message bytes duplication in Kafka headers when compression is enabled

2023-09-13 Thread Phuc Hong Tran (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764956#comment-17764956 ] Phuc Hong Tran edited comment on KAFKA-15160 at 9/14/23 3:32 AM: -

[jira] [Commented] (KAFKA-15160) Message bytes duplication in Kafka headers when compression is enabled

2023-09-13 Thread Phuc Hong Tran (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764956#comment-17764956 ] Phuc Hong Tran commented on KAFKA-15160: [~vikashmishra0808] How did you setup ConsumerRecord

[GitHub] [kafka] showuon commented on pull request #14381: KAFKA-14912:Add a dynamic config for remote index cache size

2023-09-13 Thread via GitHub
showuon commented on PR #14381: URL: https://github.com/apache/kafka/pull/14381#issuecomment-1718627959 @satishd , not sure if you want to have another look. If no, I'll merge it. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [kafka] showuon commented on pull request #14381: KAFKA-14912:Add a dynamic config for remote index cache size

2023-09-13 Thread via GitHub
showuon commented on PR #14381: URL: https://github.com/apache/kafka/pull/14381#issuecomment-1718627359 Failed tests are unrelated: ``` Build / JDK 8 and Scala 2.12 / kafka.network.SocketServerTest.testSaslReauthenticationFailureNoKip152SaslAuthenticate() Build / JDK 8 and

[GitHub] [kafka] showuon commented on pull request #14347: KAFKA-15439: Transactions test with tiered storage

2023-09-13 Thread via GitHub
showuon commented on PR #14347: URL: https://github.com/apache/kafka/pull/14347#issuecomment-1718626560 Backported to 3.6 branch. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [kafka] showuon merged pull request #14347: KAFKA-15439: Transactions test with tiered storage

2023-09-13 Thread via GitHub
showuon merged PR #14347: URL: https://github.com/apache/kafka/pull/14347 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] showuon commented on pull request #14347: KAFKA-15439: Transactions test with tiered storage

2023-09-13 Thread via GitHub
showuon commented on PR #14347: URL: https://github.com/apache/kafka/pull/14347#issuecomment-1718624599 Failed tests are unrelated: ``` Build / JDK 20 and Scala 2.13 /

[jira] [Updated] (KAFKA-15466) Add KIP-919 support for some admin APIs

2023-09-13 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe updated KAFKA-15466: - Description: Add KIP-919 support for kafka-features.sh, metadata-quorum.sh, and

[jira] [Updated] (KAFKA-15466) Add KIP-919 support for some admin APIs

2023-09-13 Thread Colin McCabe (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin McCabe updated KAFKA-15466: - Summary: Add KIP-919 support for some admin APIs (was: Add KIP-919 support to

[jira] [Created] (KAFKA-15466) Add KIP-919 support to kafka-features.sh, kafka-metadata-quorum.sh, kafka-cluster.sh

2023-09-13 Thread Colin McCabe (Jira)
Colin McCabe created KAFKA-15466: Summary: Add KIP-919 support to kafka-features.sh, kafka-metadata-quorum.sh, kafka-cluster.sh Key: KAFKA-15466 URL: https://issues.apache.org/jira/browse/KAFKA-15466

[GitHub] [kafka] philipnee commented on a diff in pull request #14386: KAFKA-14960: TopicMetadata request manager

2023-09-13 Thread via GitHub
philipnee commented on code in PR #14386: URL: https://github.com/apache/kafka/pull/14386#discussion_r1325175686 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -381,41 +377,41 @@ public String toString() { } /**

[GitHub] [kafka] kirktrue commented on a diff in pull request #14359: KAFKA-14274 [2-5/7]: Introduction of more infrastructure for forthcoming fetch request manager

2023-09-13 Thread via GitHub
kirktrue commented on code in PR #14359: URL: https://github.com/apache/kafka/pull/14359#discussion_r1325174944 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/NodeStatusDetector.java: ## @@ -0,0 +1,58 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [kafka] kirktrue commented on a diff in pull request #14359: KAFKA-14274 [2-5/7]: Introduction of more infrastructure for forthcoming fetch request manager

2023-09-13 Thread via GitHub
kirktrue commented on code in PR #14359: URL: https://github.com/apache/kafka/pull/14359#discussion_r1325174579 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java: ## @@ -91,23 +105,70 @@ public void onFailure(RuntimeException e) {

[GitHub] [kafka] kirktrue commented on a diff in pull request #14359: KAFKA-14274 [2-5/7]: Introduction of more infrastructure for forthcoming fetch request manager

2023-09-13 Thread via GitHub
kirktrue commented on code in PR #14359: URL: https://github.com/apache/kafka/pull/14359#discussion_r1325171010 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/NetworkClientDelegate.java: ## @@ -63,6 +68,31 @@ public NetworkClientDelegate(

[GitHub] [kafka] junrao commented on a diff in pull request #14386: KAFKA-14960: TopicMetadata request manager

2023-09-13 Thread via GitHub
junrao commented on code in PR #14386: URL: https://github.com/apache/kafka/pull/14386#discussion_r1325040397 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java: ## @@ -91,7 +89,7 @@ public CommitRequestManager( } /** -

[GitHub] [kafka] philipnee commented on a diff in pull request #14364: KAFKA-15278: Implement HeartbeatRequestManager to handle heartbeat requests

2023-09-13 Thread via GitHub
philipnee commented on code in PR #14364: URL: https://github.com/apache/kafka/pull/14364#discussion_r1325138384 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AssignorSelection.java: ## @@ -79,6 +79,9 @@ public int hashCode() { @Override

[GitHub] [kafka] satishd commented on a diff in pull request #14382: KAFKA-15442: add a section in doc for tiered storage

2023-09-13 Thread via GitHub
satishd commented on code in PR #14382: URL: https://github.com/apache/kafka/pull/14382#discussion_r1324263153 ## docs/ops.html: ## @@ -3859,6 +3859,80 @@ Finalizing the migration # Other configs ... + +6.11 Tiered Storage + +Tiered Storage overview + +Kafka data is

[GitHub] [kafka] satishd commented on a diff in pull request #14382: KAFKA-15442: add a section in doc for tiered storage

2023-09-13 Thread via GitHub
satishd commented on code in PR #14382: URL: https://github.com/apache/kafka/pull/14382#discussion_r1324263153 ## docs/ops.html: ## @@ -3859,6 +3859,80 @@ Finalizing the migration # Other configs ... + +6.11 Tiered Storage + +Tiered Storage overview + +Kafka data is

[jira] [Resolved] (KAFKA-15459) Convert coordinator retriable errors to a known producer response error.

2023-09-13 Thread Justine Olshan (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justine Olshan resolved KAFKA-15459. Resolution: Fixed > Convert coordinator retriable errors to a known producer response

[GitHub] [kafka] jolshan merged pull request #14378: KAFKA-15459: Convert coordinator retriable errors to a known producer response error

2023-09-13 Thread via GitHub
jolshan merged PR #14378: URL: https://github.com/apache/kafka/pull/14378 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] jolshan commented on pull request #14378: KAFKA-15459: Convert coordinator retriable errors to a known producer response error

2023-09-13 Thread via GitHub
jolshan commented on PR #14378: URL: https://github.com/apache/kafka/pull/14378#issuecomment-1718329367 Test failures look unrelated. I synced with Jason offline and he said we can proceed with the merge. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [kafka] jeffkbkim opened a new pull request, #14387: [DRAFT] KAFKA-14519: New coordinator metrics

2023-09-13 Thread via GitHub
jeffkbkim opened a new pull request, #14387: URL: https://github.com/apache/kafka/pull/14387 *More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.* *Summary of

[GitHub] [kafka] kirktrue commented on a diff in pull request #14359: KAFKA-14274 [2-5/7]: Introduction of more infrastructure for forthcoming fetch request manager

2023-09-13 Thread via GitHub
kirktrue commented on code in PR #14359: URL: https://github.com/apache/kafka/pull/14359#discussion_r1325061757 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractFetch.java: ## @@ -256,138 +258,23 @@ protected FetchRequest.Builder

[GitHub] [kafka] kirktrue commented on a diff in pull request #14359: KAFKA-14274 [2-5/7]: Introduction of more infrastructure for forthcoming fetch request manager

2023-09-13 Thread via GitHub
kirktrue commented on code in PR #14359: URL: https://github.com/apache/kafka/pull/14359#discussion_r1325058583 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/FetchUtils.java: ## @@ -0,0 +1,53 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [kafka] kirktrue commented on a diff in pull request #14359: KAFKA-14274 [2-5/7]: Introduction of more infrastructure for forthcoming fetch request manager

2023-09-13 Thread via GitHub
kirktrue commented on code in PR #14359: URL: https://github.com/apache/kafka/pull/14359#discussion_r1325056350 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/FetchBuffer.java: ## @@ -0,0 +1,143 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[jira] [Commented] (KAFKA-15465) MM2 not working when its internal topics are pre-created on a cluster that disallows topic creation

2023-09-13 Thread Ahmed HIBOT (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764867#comment-17764867 ] Ahmed HIBOT commented on KAFKA-15465: - [~omnia_h_ibrahim], Can you please take a look and confirm my

[jira] [Updated] (KAFKA-15465) MM2 not working when its internal topics are pre-created on a cluster that disallows topic creation

2023-09-13 Thread Ahmed HIBOT (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmed HIBOT updated KAFKA-15465: Description: h1. Replication steps * Setup a source kafka cluster (alias SOURCE) which doesn't

[GitHub] [kafka] kirktrue commented on a diff in pull request #14359: KAFKA-14274 [2-5/7]: Introduction of more infrastructure for forthcoming fetch request manager

2023-09-13 Thread via GitHub
kirktrue commented on code in PR #14359: URL: https://github.com/apache/kafka/pull/14359#discussion_r1325038436 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/FetchBuffer.java: ## @@ -0,0 +1,143 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[jira] [Created] (KAFKA-15465) MM2 not working when its internal topics are pre-created on a cluster that disallows topic creation

2023-09-13 Thread Ahmed HIBOT (Jira)
Ahmed HIBOT created KAFKA-15465: --- Summary: MM2 not working when its internal topics are pre-created on a cluster that disallows topic creation Key: KAFKA-15465 URL: https://issues.apache.org/jira/browse/KAFKA-15465

[GitHub] [kafka] mumrah commented on a diff in pull request #14376: KAFKA-15458: Fully resolve endpoint information before registering controllers

2023-09-13 Thread via GitHub
mumrah commented on code in PR #14376: URL: https://github.com/apache/kafka/pull/14376#discussion_r1324939918 ## metadata/src/main/java/org/apache/kafka/metadata/ListenerInfo.java: ## @@ -0,0 +1,376 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more +

[GitHub] [kafka] philipnee opened a new pull request, #14386: KAFKA-14960: TopicMetadata request manager

2023-09-13 Thread via GitHub
philipnee opened a new pull request, #14386: URL: https://github.com/apache/kafka/pull/14386 TopicMetadataRequestManager is responsible for sending topic metadata requests. The manager manages API requests and build the request accordingly. All topic metadata requests are chained, if

[GitHub] [kafka] philipnee closed pull request #14362: KAFKA-14960: TopicMetadata request manager

2023-09-13 Thread via GitHub
philipnee closed pull request #14362: KAFKA-14960: TopicMetadata request manager URL: https://github.com/apache/kafka/pull/14362 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [kafka] lianetm opened a new pull request, #14385: KAFKA-15306 - Integrating committed offsets for updating fetch positions

2023-09-13 Thread via GitHub
lianetm opened a new pull request, #14385: URL: https://github.com/apache/kafka/pull/14385 Support for using committed offsets to update fetch positions. This PR includes: - moving`refreshCommittedOffsets` function out of the existing `ConsumerCoordinator` so it can be reused (no

[jira] [Assigned] (KAFKA-15464) Allow dynamic reloading of certificates with different DN / SANs

2023-09-13 Thread Jakub Scholz (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakub Scholz reassigned KAFKA-15464: Assignee: Jakub Scholz > Allow dynamic reloading of certificates with different DN /

[jira] [Created] (KAFKA-15464) Allow dynamic reloading of certificates with different DN / SANs

2023-09-13 Thread Jakub Scholz (Jira)
Jakub Scholz created KAFKA-15464: Summary: Allow dynamic reloading of certificates with different DN / SANs Key: KAFKA-15464 URL: https://issues.apache.org/jira/browse/KAFKA-15464 Project: Kafka

[jira] [Commented] (KAFKA-15452) Custom KafkaPrincipalBuilder Cannot Access SslPrincipalMapper

2023-09-13 Thread Raghu Baddam (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764839#comment-17764839 ] Raghu Baddam commented on KAFKA-15452: -- [~hachikuji] /[~mimaison] /[~mjsax]: Do you mind validating

[GitHub] [kafka] zhaohaidao commented on pull request #14271: KAFKA-14503: Implement ListGroups

2023-09-13 Thread via GitHub
zhaohaidao commented on PR #14271: URL: https://github.com/apache/kafka/pull/14271#issuecomment-1718110509 > @zhaohaidao Thanks for all the work you are doing on this. I really appreciate it. I have created a few jiras for the next steps: > > *

[GitHub] [kafka] splett2 commented on a diff in pull request #14053: KAFKA-15221; Fix the race between fetch requests from a rebooted follower.

2023-09-13 Thread via GitHub
splett2 commented on code in PR #14053: URL: https://github.com/apache/kafka/pull/14053#discussion_r1324902142 ## core/src/main/scala/kafka/cluster/Partition.scala: ## @@ -864,13 +866,19 @@ class Partition(val topicPartition: TopicPartition, // No need to calculate low

[GitHub] [kafka] zhaohaidao commented on pull request #14271: KAFKA-14503: Implement ListGroups

2023-09-13 Thread via GitHub
zhaohaidao commented on PR #14271: URL: https://github.com/apache/kafka/pull/14271#issuecomment-1718105046 > @zhaohaidao Thanks for the update. I made another pass on it and I left a bunch of small comments. We should be able to merge it when they are addressed. Could you also rebase the

[GitHub] [kafka] zhaohaidao commented on a diff in pull request #14271: KAFKA-14503: Implement ListGroups

2023-09-13 Thread via GitHub
zhaohaidao commented on code in PR #14271: URL: https://github.com/apache/kafka/pull/14271#discussion_r1324884926 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -431,14 +434,17 @@ public void rollback() {

[GitHub] [kafka] zhaohaidao commented on a diff in pull request #14271: KAFKA-14503: Implement ListGroups

2023-09-13 Thread via GitHub
zhaohaidao commented on code in PR #14271: URL: https://github.com/apache/kafka/pull/14271#discussion_r1324884926 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -431,14 +434,17 @@ public void rollback() {

[GitHub] [kafka] jolshan commented on a diff in pull request #14378: KAFKA-15459: Convert coordinator retriable errors to a known producer response error

2023-09-13 Thread via GitHub
jolshan commented on code in PR #14378: URL: https://github.com/apache/kafka/pull/14378#discussion_r1324872270 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -753,39 +753,63 @@ class ReplicaManager(val config: KafkaConfig, val localProduceResults =

[GitHub] [kafka] dajac commented on a diff in pull request #14378: KAFKA-15459: Convert coordinator retriable errors to a known producer response error

2023-09-13 Thread via GitHub
dajac commented on code in PR #14378: URL: https://github.com/apache/kafka/pull/14378#discussion_r1324871934 ## core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala: ## @@ -2569,6 +2571,70 @@ class ReplicaManagerTest { assertEquals((Errors.NONE, node0),

[GitHub] [kafka] zhaohaidao commented on a diff in pull request #14271: KAFKA-14503: Implement ListGroups

2023-09-13 Thread via GitHub
zhaohaidao commented on code in PR #14271: URL: https://github.com/apache/kafka/pull/14271#discussion_r1324870944 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupCoordinatorServiceTest.java: ## @@ -599,6 +608,164 @@ public void

[GitHub] [kafka] jolshan commented on a diff in pull request #14378: KAFKA-15459: Convert coordinator retriable errors to a known producer response error

2023-09-13 Thread via GitHub
jolshan commented on code in PR #14378: URL: https://github.com/apache/kafka/pull/14378#discussion_r1324870442 ## core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala: ## @@ -2569,6 +2571,70 @@ class ReplicaManagerTest { assertEquals((Errors.NONE, node0),

[GitHub] [kafka] dajac commented on a diff in pull request #14378: KAFKA-15459: Convert coordinator retriable errors to a known producer response error

2023-09-13 Thread via GitHub
dajac commented on code in PR #14378: URL: https://github.com/apache/kafka/pull/14378#discussion_r1324868208 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -753,39 +753,63 @@ class ReplicaManager(val config: KafkaConfig, val localProduceResults =

[GitHub] [kafka] dajac commented on a diff in pull request #14378: KAFKA-15459: Convert coordinator retriable errors to a known producer response error

2023-09-13 Thread via GitHub
dajac commented on code in PR #14378: URL: https://github.com/apache/kafka/pull/14378#discussion_r1324869043 ## core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala: ## @@ -2569,6 +2571,70 @@ class ReplicaManagerTest { assertEquals((Errors.NONE, node0),

[GitHub] [kafka] dajac commented on a diff in pull request #14378: KAFKA-15459: Convert coordinator retriable errors to a known producer response error

2023-09-13 Thread via GitHub
dajac commented on code in PR #14378: URL: https://github.com/apache/kafka/pull/14378#discussion_r1324868507 ## core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala: ## @@ -2569,6 +2571,70 @@ class ReplicaManagerTest { assertEquals((Errors.NONE, node0),

[GitHub] [kafka] dajac commented on a diff in pull request #14378: KAFKA-15459: Convert coordinator retriable errors to a known producer response error

2023-09-13 Thread via GitHub
dajac commented on code in PR #14378: URL: https://github.com/apache/kafka/pull/14378#discussion_r1324868208 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -753,39 +753,63 @@ class ReplicaManager(val config: KafkaConfig, val localProduceResults =

[GitHub] [kafka] zhaohaidao commented on a diff in pull request #14271: KAFKA-14503: Implement ListGroups

2023-09-13 Thread via GitHub
zhaohaidao commented on code in PR #14271: URL: https://github.com/apache/kafka/pull/14271#discussion_r1324869392 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -8574,6 +8584,93 @@ public void

[GitHub] [kafka] dajac commented on a diff in pull request #14378: KAFKA-15459: Convert coordinator retriable errors to a known producer response error

2023-09-13 Thread via GitHub
dajac commented on code in PR #14378: URL: https://github.com/apache/kafka/pull/14378#discussion_r1324869297 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -753,39 +753,56 @@ class ReplicaManager(val config: KafkaConfig, val localProduceResults =

[GitHub] [kafka] zhaohaidao commented on a diff in pull request #14271: KAFKA-14503: Implement ListGroups

2023-09-13 Thread via GitHub
zhaohaidao commented on code in PR #14271: URL: https://github.com/apache/kafka/pull/14271#discussion_r1324869129 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupMetadataManagerTest.java: ## @@ -8574,6 +8584,93 @@ public void

[GitHub] [kafka] splett2 commented on a diff in pull request #14053: KAFKA-15221; Fix the race between fetch requests from a rebooted follower.

2023-09-13 Thread via GitHub
splett2 commented on code in PR #14053: URL: https://github.com/apache/kafka/pull/14053#discussion_r1324845475 ## core/src/main/scala/kafka/cluster/Partition.scala: ## @@ -137,7 +137,8 @@ object Partition { delayedOperations = delayedOperations, metadataCache =

[jira] [Updated] (KAFKA-15463) StreamsException: Accessing from an unknown node

2023-09-13 Thread Matthias J. Sax (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax updated KAFKA-15463: Priority: Major (was: Blocker) > StreamsException: Accessing from an unknown node >

[GitHub] [kafka] mjsax commented on a diff in pull request #14322: KAFKA-15418: update statement on decompression

2023-09-13 Thread via GitHub
mjsax commented on code in PR #14322: URL: https://github.com/apache/kafka/pull/14322#discussion_r1324811185 ## docs/design.html: ## @@ -136,8 +136,10 @@ -Kafka supports this with an efficient batching format. A batch of messages can be clumped together compressed and

[GitHub] [kafka] davetroiano commented on a diff in pull request #14322: KAFKA-15418: update statement on decompression

2023-09-13 Thread via GitHub
davetroiano commented on code in PR #14322: URL: https://github.com/apache/kafka/pull/14322#discussion_r1324799715 ## docs/design.html: ## @@ -136,8 +136,10 @@ -Kafka supports this with an efficient batching format. A batch of messages can be clumped together compressed

[GitHub] [kafka] jolshan commented on a diff in pull request #14378: KAFKA-15459: Convert coordinator retriable errors to a known producer response error

2023-09-13 Thread via GitHub
jolshan commented on code in PR #14378: URL: https://github.com/apache/kafka/pull/14378#discussion_r1324792282 ## core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala: ## @@ -2569,6 +2571,70 @@ class ReplicaManagerTest { assertEquals((Errors.NONE, node0),

[GitHub] [kafka] zhaohaidao commented on a diff in pull request #14271: KAFKA-14503: Implement ListGroups

2023-09-13 Thread via GitHub
zhaohaidao commented on code in PR #14271: URL: https://github.com/apache/kafka/pull/14271#discussion_r1324791489 ## group-coordinator/src/test/java/org/apache/kafka/coordinator/group/GroupCoordinatorServiceTest.java: ## @@ -599,6 +608,164 @@ public void

[GitHub] [kafka] junrao commented on a diff in pull request #14359: KAFKA-14274 [2-5/7]: Introduction of more infrastructure for forthcoming fetch request manager

2023-09-13 Thread via GitHub
junrao commented on code in PR #14359: URL: https://github.com/apache/kafka/pull/14359#discussion_r1323771865 ## clients/src/main/java/org/apache/kafka/clients/consumer/internals/NetworkClientDelegate.java: ## @@ -63,6 +68,31 @@ public NetworkClientDelegate(

[GitHub] [kafka] OmniaGM commented on pull request #13204: KAFKA-14593: Move LeaderElectionCommand to tools

2023-09-13 Thread via GitHub
OmniaGM commented on PR #13204: URL: https://github.com/apache/kafka/pull/13204#issuecomment-1717905593 I found out why the tests were failing. It turns out that we are hitting a similar problem to this gradle [issue#847](https://github.com/gradle/gradle/issues/847) that causing

[GitHub] [kafka] OmniaGM commented on pull request #13201: KAFKA-14596: Move TopicCommand to tools

2023-09-13 Thread via GitHub
OmniaGM commented on PR #13201: URL: https://github.com/apache/kafka/pull/13201#issuecomment-1717877198 > Hi @OmniaGM, thanks. I left some comments. > > Additionally, this error message is different from previous implementation: > > ```shell > ### OLD > $

[GitHub] [kafka] jolshan commented on a diff in pull request #14378: KAFKA-15459: Convert coordinator retriable errors to a known producer response error

2023-09-13 Thread via GitHub
jolshan commented on code in PR #14378: URL: https://github.com/apache/kafka/pull/14378#discussion_r1324699341 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -753,39 +753,63 @@ class ReplicaManager(val config: KafkaConfig, val localProduceResults =

[GitHub] [kafka] jolshan commented on a diff in pull request #14378: KAFKA-15459: Convert coordinator retriable errors to a known producer response error

2023-09-13 Thread via GitHub
jolshan commented on code in PR #14378: URL: https://github.com/apache/kafka/pull/14378#discussion_r1324697600 ## core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala: ## @@ -2569,6 +2571,70 @@ class ReplicaManagerTest { assertEquals((Errors.NONE, node0),

[GitHub] [kafka] jolshan commented on a diff in pull request #14378: KAFKA-15459: Convert coordinator retriable errors to a known producer response error

2023-09-13 Thread via GitHub
jolshan commented on code in PR #14378: URL: https://github.com/apache/kafka/pull/14378#discussion_r1324696082 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -753,39 +753,63 @@ class ReplicaManager(val config: KafkaConfig, val localProduceResults =

[GitHub] [kafka] jolshan commented on a diff in pull request #14378: KAFKA-15459: Convert coordinator retriable errors to a known producer response error

2023-09-13 Thread via GitHub
jolshan commented on code in PR #14378: URL: https://github.com/apache/kafka/pull/14378#discussion_r1324692053 ## core/src/test/scala/unit/kafka/server/ReplicaManagerTest.scala: ## @@ -2569,6 +2571,70 @@ class ReplicaManagerTest { assertEquals((Errors.NONE, node0),

[jira] [Resolved] (KAFKA-15163) Implement validatePositions functionality for new KafkaConsumer

2023-09-13 Thread Jun Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-15163. - Fix Version/s: 3.7.0 Resolution: Fixed This is covered in

[GitHub] [kafka] AndrewJSchofield commented on a diff in pull request #14384: KAFKA-15415 On producer-batch retry, skip-backoff on a new leader

2023-09-13 Thread via GitHub
AndrewJSchofield commented on code in PR #14384: URL: https://github.com/apache/kafka/pull/14384#discussion_r1324590647 ## clients/src/main/java/org/apache/kafka/common/PartitionInfo.java: ## @@ -60,12 +69,19 @@ public int partition() { } /** - * The node id of

[jira] [Resolved] (KAFKA-15115) Implement resetPositions functionality in OffsetsRequestManager

2023-09-13 Thread Jun Rao (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-15115. - Fix Version/s: 3.7.0 Resolution: Fixed merged the PR to trunk > Implement resetPositions

[GitHub] [kafka] junrao merged pull request #14346: KAFKA-15115 - KAFKA-15163; Reset/Validate positions implementation & API integration

2023-09-13 Thread via GitHub
junrao merged PR #14346: URL: https://github.com/apache/kafka/pull/14346 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [kafka] tylerbertrand commented on a diff in pull request #14344: Resolve checkstyle cache miss

2023-09-13 Thread via GitHub
tylerbertrand commented on code in PR #14344: URL: https://github.com/apache/kafka/pull/14344#discussion_r1324625070 ## build.gradle: ## @@ -685,6 +685,7 @@ subprojects { } checkstyle { +configDirectory = rootProject.layout.projectDirectory.dir("checkstyle") Review

[GitHub] [kafka] tylerbertrand commented on a diff in pull request #14344: Resolve checkstyle cache miss

2023-09-13 Thread via GitHub
tylerbertrand commented on code in PR #14344: URL: https://github.com/apache/kafka/pull/14344#discussion_r1324618554 ## build.gradle: ## @@ -685,6 +685,7 @@ subprojects { } checkstyle { +configDirectory = rootProject.layout.projectDirectory.dir("checkstyle")

[GitHub] [kafka] tylerbertrand commented on a diff in pull request #14344: Resolve checkstyle cache miss

2023-09-13 Thread via GitHub
tylerbertrand commented on code in PR #14344: URL: https://github.com/apache/kafka/pull/14344#discussion_r1324618554 ## build.gradle: ## @@ -685,6 +685,7 @@ subprojects { } checkstyle { +configDirectory = rootProject.layout.projectDirectory.dir("checkstyle")

[GitHub] [kafka] satishd commented on pull request #14382: KAFKA-15442: add a section in doc for tiered storage

2023-09-13 Thread via GitHub
satishd commented on PR #14382: URL: https://github.com/apache/kafka/pull/14382#issuecomment-1717677707 >That sounds good. But since we don't have much time left for v3.6.0 RC build, we can add them after release. Doc updates can directly update in kafka-site repo. I'll add them in a

[jira] [Commented] (KAFKA-15160) Message bytes duplication in Kafka headers when compression is enabled

2023-09-13 Thread Phuc Hong Tran (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17764677#comment-17764677 ] Phuc Hong Tran commented on KAFKA-15160: [~vikashmishra0808] Sorry for the delay, I'm

[GitHub] [kafka] lianetm commented on pull request #14346: KAFKA-15115 - KAFKA-15163; Reset/Validate positions implementation & API integration

2023-09-13 Thread via GitHub
lianetm commented on PR #14346: URL: https://github.com/apache/kafka/pull/14346#issuecomment-1717579385 There were 6 unrelated test failures in the last build: Build / JDK 20 and Scala 2.13 / testTaskRequestWithOldStartMsGetsUpdated() –

[GitHub] [kafka] cadonna commented on a diff in pull request #14317: KAFKA-13973: Fix inflated block cache metrics

2023-09-13 Thread via GitHub
cadonna commented on code in PR #14317: URL: https://github.com/apache/kafka/pull/14317#discussion_r1324450860 ## streams/src/test/java/org/apache/kafka/streams/state/internals/metrics/RocksDBBlockCacheMetricsTest.java: ## @@ -0,0 +1,133 @@ +/* + * Licensed to the Apache

[GitHub] [kafka] showuon commented on pull request #14382: KAFKA-15442: add a section in doc for tiered storage

2023-09-13 Thread via GitHub
showuon commented on PR #14382: URL: https://github.com/apache/kafka/pull/14382#issuecomment-1717567763 > Will the RemoteLogManagerConfig and TBRLMMConfig will be auto-exported in the doc config section? Yes, that's been handled in another

[GitHub] [kafka] dajac commented on a diff in pull request #14378: KAFKA-15459: Convert coordinator retriable errors to a known producer response error

2023-09-13 Thread via GitHub
dajac commented on code in PR #14378: URL: https://github.com/apache/kafka/pull/14378#discussion_r1324437789 ## core/src/main/scala/kafka/server/ReplicaManager.scala: ## @@ -753,39 +753,63 @@ class ReplicaManager(val config: KafkaConfig, val localProduceResults =

[GitHub] [kafka] showuon commented on a diff in pull request #14382: KAFKA-15442: add a section in doc for tiered storage

2023-09-13 Thread via GitHub
showuon commented on code in PR #14382: URL: https://github.com/apache/kafka/pull/14382#discussion_r1324438818 ## docs/ops.html: ## @@ -3859,6 +3859,80 @@ Finalizing the migration # Other configs ... + +6.11 Tiered Storage + +Tiered Storage overview + +Kafka data is

  1   2   >