Re: [DISCUSS] KIP-1124: Providing a clear Kafka Client upgrade path

Thanks all. I am happy that we are making progress to close this out. As mentioned in my (long) email last Friday, I also think that the KIP makes a lot of implicit assumptions, and it might benefit from begin more explicit, even explaining things that are "set in stone" already, and the KIP d

Re: [VOTE] KIP-1111: Enforcing Explicit Naming for Kafka Streams Internal Topics

Thanks for pointing this out. I don't have any concerns. Also, even if the KIP make this statement, logging is usually treated more as an implementation detail. Guess, that is why I did not pay too much attention to it to begin with. Thus, if it would really become a problem, we can always ju

Re: [VOTE] 4.0.0 RC0

Hi David, I focused on testing the clients using Java 11: My steps: 1. Start the broker via the native Docker image (apache/kafka-native:4.0.0-rc0) 2. Wrote a simple client to write and read messages, using kafka-clients 4.0.0 from https://repository.apache.org/content/groups/staging 3. Ran my

[jira] [Resolved] (KAFKA-18860) Clarify the KRaft missing feature

[ https://issues.apache.org/jira/browse/KAFKA-18860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-18860. Resolution: Fixed trunk: [https://github.com/apache/kafka/commit/e2d9ced098c35068900746af

Re: [DISCUSS] Apache Kafka 4.0.0 release

During testing RC0, Bruno found that Kafka Streams examples require Java 17 right now, while Kafka Streams itself still supports Java 11. I opened a PR to fix this: https://github.com/apache/kafka/pull/19052 We hope it can be considered for 4.0 release. -Matthias On 2/20/25 11:23 PM, David J

Re: [DISCUSS] KIP-1131: Controller-side monitoring for broker shutdown and startup

> > I guess my concern is that the time-based metrics would reset to 0 on > every failover (if I understand the proposed implementation correctly). > That seems likely to create confusion. Yeah that makes sense to me. I'm fine with moving towards the approach of either (since I don't think we need

[jira] [Created] (KAFKA-18898) 4.0 Upgrade docs rendering below other versions

David Arthur created KAFKA-18898: Summary: 4.0 Upgrade docs rendering below other versions Key: KAFKA-18898 URL: https://issues.apache.org/jira/browse/KAFKA-18898 Project: Kafka Issue Type: B

[jira] [Created] (KAFKA-18891) Add support for RemoteLogMetadataManager and RemoteStorageManager

Mickael Maison created KAFKA-18891: -- Summary: Add support for RemoteLogMetadataManager and RemoteStorageManager Key: KAFKA-18891 URL: https://issues.apache.org/jira/browse/KAFKA-18891 Project: Kafka

[jira] [Created] (KAFKA-18897) Add support for SslEngineFactory

Mickael Maison created KAFKA-18897: -- Summary: Add support for SslEngineFactory Key: KAFKA-18897 URL: https://issues.apache.org/jira/browse/KAFKA-18897 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-18896) Add support for Login

Mickael Maison created KAFKA-18896: -- Summary: Add support for Login Key: KAFKA-18896 URL: https://issues.apache.org/jira/browse/KAFKA-18896 Project: Kafka Issue Type: Sub-task Re

[jira] [Created] (KAFKA-18895) Add support for AuthenticateCallbackHandler

Mickael Maison created KAFKA-18895: -- Summary: Add support for AuthenticateCallbackHandler Key: KAFKA-18895 URL: https://issues.apache.org/jira/browse/KAFKA-18895 Project: Kafka Issue Type: S

[jira] [Created] (KAFKA-18894) Add support for ConfigProvider

Mickael Maison created KAFKA-18894: -- Summary: Add support for ConfigProvider Key: KAFKA-18894 URL: https://issues.apache.org/jira/browse/KAFKA-18894 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-18893) Add support for ReplicaSelector

Mickael Maison created KAFKA-18893: -- Summary: Add support for ReplicaSelector Key: KAFKA-18893 URL: https://issues.apache.org/jira/browse/KAFKA-18893 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-18892) Add support for ClientQuotaCallback

Mickael Maison created KAFKA-18892: -- Summary: Add support for ClientQuotaCallback Key: KAFKA-18892 URL: https://issues.apache.org/jira/browse/KAFKA-18892 Project: Kafka Issue Type: Sub-task

[jira] [Created] (KAFKA-18890) Add support for CreateTopicPolicy and AlterConfigPolicy

Mickael Maison created KAFKA-18890: -- Summary: Add support for CreateTopicPolicy and AlterConfigPolicy Key: KAFKA-18890 URL: https://issues.apache.org/jira/browse/KAFKA-18890 Project: Kafka I

Re: Metadata requests for subset of partitions

Hi Michał, On Thu, Feb 27, 2025, at 3:44 AM, Michał Łowicki wrote: > Hi there! > > Is there any reason why Metadata requests > do not > support fetching metadata for subsets of the partitions? If a certain > client is interested only

Re: Proposal: Improving PR Review Process

Nelson, thanks for reaching out and for the suggestions. We're pretty aware of the problem and have taken some steps lately to improve the process. A few highlights: * We used to have over 1000 PRs open. We have reduced this down to the 200s by closing stale PRs * We now add a "triage" label to a

Re: Support for other OAuth2 grant types in Kafka

Hi Subra, On Thu, Feb 27, 2025, at 5:41 AM, Subra I wrote: > Thanks for the response Kirk. I will look at the links shared as well. > > It is clear from the earlier links that as of now, Kafka only supports > client credentials grant type by default. Am I right? (Unless we do a > custom implement

Re: [DISCUSS] KIP-1124: Providing a clear Kafka Client upgrade path

Hi Bruno, A comment below. On Thu, Feb 27, 2025, 1:20 AM Bruno Cadonna wrote: > BC2: > Another aspect that confuses me in the KIP is the bridge version for > clients. If my broker version is 3.9.x, I can do a direct upgrade from > 1.0.x to 4.0.x (let apart the Java API compatibility mentioned a

Re: Support for other OAuth2 grant types in Kafka

Thanks for the response Kirk. I will look at the links shared as well. It is clear from the earlier links that as of now, Kafka only supports client credentials grant type by default. Am I right? (Unless we do a custom implementation as mentioned by you) On Wed, Feb 26, 2025 at 11:46 PM Kirk True

Re: [DISCUSS] KIP-1134: Virtual Clusters in Kafka

Hi Greg, Thank you for your comments! 1. > Can you provide an example of a situation where a client in a VC needs to > refer to resources in a child VC using the existing APIs? I think topic creation/topic listing are both good examples, but I'd argue that a cluster admin might need to be able

[jira] [Created] (KAFKA-18889) Make records in ShareFetchResponse non-nullable

Apoorv Mittal created KAFKA-18889: - Summary: Make records in ShareFetchResponse non-nullable Key: KAFKA-18889 URL: https://issues.apache.org/jira/browse/KAFKA-18889 Project: Kafka Issue Type:

[jira] [Created] (KAFKA-18888) Add support for Authorizer

Mickael Maison created KAFKA-1: -- Summary: Add support for Authorizer Key: KAFKA-1 URL: https://issues.apache.org/jira/browse/KAFKA-1 Project: Kafka Issue Type: Sub-task

Metadata requests for subset of partitions

Hi there! Is there any reason why Metadata requests do not support fetching metadata for subsets of the partitions? If a certain client is interested only in e.g. 1 but topic may have many so most of fetched data isn't really used.

[jira] [Created] (KAFKA-18887) Implement Admin API extensions beyond list/describe group (delete group, offset-related APIs)

Alieh Saeedi created KAFKA-18887: Summary: Implement Admin API extensions beyond list/describe group (delete group, offset-related APIs) Key: KAFKA-18887 URL: https://issues.apache.org/jira/browse/KAFKA-18887

Proposal: Improving PR Review Process

Hi all, I’d like to suggest a revision to our PR review process. Right now, it seems like the process is to tag people in comments and hope they respond. But in my experience, this can take painfully long—especially for infrequent contributors who aren’t part of any community or organization that

[jira] [Created] (KAFKA-18886) add behavior change of CreateTopicPolicy and AlterConfigPolicy to zk2kraft

Chia-Ping Tsai created KAFKA-18886: -- Summary: add behavior change of CreateTopicPolicy and AlterConfigPolicy to zk2kraft Key: KAFKA-18886 URL: https://issues.apache.org/jira/browse/KAFKA-18886 Projec

[jira] [Created] (KAFKA-18885) document the behavioral differences between ZooKeeper and KRaft

Chia-Ping Tsai created KAFKA-18885: -- Summary: document the behavioral differences between ZooKeeper and KRaft Key: KAFKA-18885 URL: https://issues.apache.org/jira/browse/KAFKA-18885 Project: Kafka

[jira] [Created] (KAFKA-18884) Move TransactionMetadata to transaction-coordinator module

PoAn Yang created KAFKA-18884: - Summary: Move TransactionMetadata to transaction-coordinator module Key: KAFKA-18884 URL: https://issues.apache.org/jira/browse/KAFKA-18884 Project: Kafka Issue Ty

[jira] [Created] (KAFKA-18883) Move TransactionLog to transaction-coordinator module

PoAn Yang created KAFKA-18883: - Summary: Move TransactionLog to transaction-coordinator module Key: KAFKA-18883 URL: https://issues.apache.org/jira/browse/KAFKA-18883 Project: Kafka Issue Type: S

[jira] [Created] (KAFKA-18882) Move BaseKey, TxnKey, and UnknownKey to transaction-coordinator module

Chia-Ping Tsai created KAFKA-18882: -- Summary: Move BaseKey, TxnKey, and UnknownKey to transaction-coordinator module Key: KAFKA-18882 URL: https://issues.apache.org/jira/browse/KAFKA-18882 Project: K

[jira] [Resolved] (KAFKA-18471) Race conditions when accessing RecordHeader data

[ https://issues.apache.org/jira/browse/KAFKA-18471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-18471. Resolution: Duplicate duplicate to KAFKA-18470 > Race conditions when accessing RecordHe

[jira] [Created] (KAFKA-18881) Document the ConsumerRecord as non-thread safe

Chia-Ping Tsai created KAFKA-18881: -- Summary: Document the ConsumerRecord as non-thread safe Key: KAFKA-18881 URL: https://issues.apache.org/jira/browse/KAFKA-18881 Project: Kafka Issue Type

Re: [DISCUSS] KIP-1124: Providing a clear Kafka Client upgrade path

Hi, Thanks for all the updates and all the discussions regarding this KIP! IMO, the KIP does not precisely enough explain the rationale behind the proposed upgrade path. That makes the KIP hard to understand. I believe the KIP should be a bit more explicit about the reasons. That does not mea

[jira] [Created] (KAFKA-18880) Remove kafka.cluster.Broker and BrokerEndPointNotAvailableException

Chia-Ping Tsai created KAFKA-18880: -- Summary: Remove kafka.cluster.Broker and BrokerEndPointNotAvailableException Key: KAFKA-18880 URL: https://issues.apache.org/jira/browse/KAFKA-18880 Project: Kafk

[jira] [Resolved] (KAFKA-18849) add "strict min ISR" to the docs of "min.insync.replicas"

[ https://issues.apache.org/jira/browse/KAFKA-18849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-18849. Resolution: Fixed trunk: [https://github.com/apache/kafka/commit/269e2d898b76b5c9c58232c3