Re: KIP-919: Allow AdminClient to Talk Directly with the KRaft Controller Quorum

Hey Colin, I still need to finish reading and understanding the KIP, but I have a couple of comments despite being ignorant of most of the KRaft stuff. (Sorry!) Firstly, does it make sense to create an extension of the current AdminClient only to handle these specific KRaft use cases? It seems

Jenkins build is unstable: Kafka » Kafka Branch Builder » trunk #1780

See

Re: KIP-919: Allow AdminClient to Talk Directly with the KRaft Controller Quorum

Hello Colin, There is a mistake that we use `—bootstrap-server` instead of `—bootstrap-server(s)`, so should we also change the new argument `—bootstrap-controller` (no s). -- Ziming > On Apr 20, 2023, at 05:17, Colin McCabe wrote: > > Hi all, > > I wrote a short KIP about allowing

Re: [DISCUSS] KIP-909: Allow clients to rebootstrap DNS lookup failure

Hey Jason, Thanks for your review. I think if we make it a retriable error, does it make sense to have a configurable timeout still? as we expect the user to continue to retry anyway. I'm considering the case of bad configuration. If the user retries the error, then we rely on the error/warning

Re: [DISCUSS] KIP-909: Allow clients to rebootstrap DNS lookup failure

Hey Phillip, The KIP looks good. 5 minutes seems like a reasonable tradeoff. I do wonder if it is necessary to treat bootstrap timeout as a fatal error though. It seems possible that the exception might be caught by handlers in existing applications which may not expect that the client needs to

Re: [DISCUSS] Re-visit end of life policy

While I understand the desire, I tend to agree with Ismael. In general, it's a significant amount of work not just to do the actual releases, but also the cherry-pick bug-fixed to older branches. Code diverges very quickly, and a clean cherry-pick is usually only possible for one or two

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #1779

See Changes: -- [...truncated 467630 lines...] [2023-04-19T23:22:42.565Z] [2023-04-19T23:22:42.565Z] Gradle Test Run :connect:runtime:integrationTest > Gradle Test

Re: KIP-919: Allow AdminClient to Talk Directly with the KRaft Controller Quorum

On Wed, Apr 19, 2023, at 14:37, Ron Dagostino wrote: > Thanks for the KIP, Colin. > > There seems to be some inconsistency between sometimes referring to > "TargetKRaftControllerQuorum" and other times referring to > "DirectToKRaftControllerQuorum". Aside from that, it looks good to > me. The

Re: KIP-919: Allow AdminClient to Talk Directly with the KRaft Controller Quorum

Thanks for the KIP, Colin. There seems to be some inconsistency between sometimes referring to "TargetKRaftControllerQuorum" and other times referring to "DirectToKRaftControllerQuorum". Aside from that, it looks good to me. The symmetry of bootstrap servers and bootstrap controllers feels

KIP-919: Allow AdminClient to Talk Directly with the KRaft Controller Quorum

Hi all, I wrote a short KIP about allowing AdminClient to talk directly with the KRaft controller quorum. Check it out here: https://cwiki.apache.org/confluence/x/Owo0Dw best, Colin

[jira] [Resolved] (KAFKA-4327) Move Reset Tool from core to streams

[ https://issues.apache.org/jira/browse/KAFKA-4327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias J. Sax resolved KAFKA-4327. Fix Version/s: (was: 4.0.0) Resolution: Fixed This was resolved via

Re: [DISCUSS] KIP-892: Transactional Semantics for StateStores

Hi Colt, The issue is that if there's a crash between 2 and 3, then you still end up with inconsistent data in RocksDB. The only way to guarantee that your checkpoint offsets and locally stored data are consistent with each other are to atomically commit them, which can be achieved by having the

[jira] [Created] (KAFKA-14922) kafka-streams-application-reset deletes topics not belonging to specified application-id

Jørgen created KAFKA-14922: -- Summary: kafka-streams-application-reset deletes topics not belonging to specified application-id Key: KAFKA-14922 URL: https://issues.apache.org/jira/browse/KAFKA-14922

Re: Adding reviewers with Github actions

It's a lot more convenient to have it in the commit than having to follow links, etc. David Arthur also wrote a script to help with this step, I believe. Ismael On Tue, Apr 18, 2023, 9:29 AM Divij Vaidya wrote: > Do we even need a manual attribution for a reviewer in the commit message? >

Re: [DISCUSS] KIP-892: Transactional Semantics for StateStores

Nick, Thanks for your reply. Ack to A) and B). For item C), I see what you're referring to. Your proposed solution will work, so no need to change it. What I was suggesting was that it might be possible to achieve this with only one column family. So long as: - No uncommitted records (i.e.

Re: [DISCUSS] KIP-892: Transactional Semantics for StateStores

Hi Colt, A. I've done my best to de-couple the StateStore stuff from the rest of the Streams engine. The fact that there will be only one ongoing (write) transaction at a time is not guaranteed by any API, and is just a consequence of the way Streams operates. To that end, I tried to ensure the

KafkaPrincipal, oauth, & JWT

I am exploring how to get roles defined via oauth authentication to be passed with the KafkaPrincipal (generated by the DefaultKafkaPrincipalBuilder) so it can be used by authorization. I know the PrincipalBuilder can be replaced with a custom implementation along with an alternative

Re: [DISCUSS] KIP-918: MM2 Topic And Group Listener

I wouldn't really include a non-existent group (same as we don't care about a non-existent topic), that doesn't really matter. I think having an existing group which doesn't have an offset to checkpoint is equivalent to a topic having no records to replicate from the monitoring perspective. I

[jira] [Created] (KAFKA-14921) Avoid non numeric values for metrics

Mickael Maison created KAFKA-14921: -- Summary: Avoid non numeric values for metrics Key: KAFKA-14921 URL: https://issues.apache.org/jira/browse/KAFKA-14921 Project: Kafka Issue Type:

Re: [DISCUSS] KIP-918: MM2 Topic And Group Listener

Thanks for your reply, Daniel. Regarding the group list, do you mean that if the group of the source cluster has not committed an offset (the group does not exist or the group has not committed an offset to the topic being replicated), then the current metric cannot be collected? Then this

Re: Re: [DISCUSS] KIP-918: MM2 Topic And Group Listener

Hi hudeqi, Thank you for your comments! Related to the topic list: you are correct, the partition metrics are created eagerly, so even if a topic has no active traffic, the metrics are visible. I missed this fact when creating the KIP. For the group list: the group metrics are created lazily, so