Jenkins build is unstable: Kafka » Kafka Branch Builder » trunk #2003

See

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #2002

See Changes: -- [...truncated 386898 lines...] Gradle Test Run :streams:integrationTest > Gradle Test Executor 182 > TableTableJoinIntegrationTest > [caching enabled =

[jira] [Created] (KAFKA-15192) Network thread receives exception when updating request metrics

David Mao created KAFKA-15192: - Summary: Network thread receives exception when updating request metrics Key: KAFKA-15192 URL: https://issues.apache.org/jira/browse/KAFKA-15192 Project: Kafka

Re: [DISCUSS] KIP-951: Leader discovery optimisations for the client

Hey Mayank, Thanks for the KIP. I think this is a great proposal, and I'm in favor of this idea. A few comments: 1. Claiming metadata refresh is done asynchronously is misleading. The metadata refresh requires Network Client to be physically polled, which is done in a separate thread in

Jenkins build is unstable: Kafka » Kafka Branch Builder » trunk #2001

See

Re: KafkaConsumer refactor proposal

Hi Erik, Regarding the consumer refactor project, we’re in the process of converting Philip’s design to a “proper” KIP here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-945%3A+Update+threading+model+for+Consumer It’s still very much a draft and not ready for a formal DISCUSS

Re: KafkaConsumer refactor proposal

Hi Erik, Thanks for your interest in the client world :) I agree that the current implementation of how we handle callbacks is problematic in that it introduces race conditions and/or bottlenecks. I don’t have as much experience with the callbacks from an application developer standpoint. Is

Re: [VOTE] KIP-944 Support async runtimes in consumer, votes needed!

Hi Erik, Thanks for the KIP! I empathize with your frustration over the radio silence. It gets like that sometimes, and I apologize for my lack of feedback. I’d personally like to see this lively exchange move over to the DISCUSS thread you’d created before. Thanks, Kirk > On Jul 14, 2023,

Re: [DISCUSS] KIP-951: Leader discovery optimisations for the client

Hi Mayank, > On Jul 14, 2023, at 11:25 AM, Mayank Shekhar Narula > wrote: > > Kirk > > >> Is the requested restructuring of the response “simply” to preserve bytes, >> or is it possible that the fetch response could/should/would return >> leadership changes for partitions that we’re

Re: [DISCUSS] KIP-951: Leader discovery optimisations for the client

Hi Andrew, Good point. Sorry to be dim bulb, but I’m still not sure I understand the downsides of bumping the version. The broker and all client implementations would have to change to fully support this feature anyway, but down version clients are handled by brokers already. Thanks, Kirk >

Re: [DISCUSS] KIP-951: Leader discovery optimisations for the client

Kirk > Is the requested restructuring of the response “simply” to preserve bytes, > or is it possible that the fetch response could/should/would return > leadership changes for partitions that we’re specifically requested? > Moving endpoints to top-level fields would preserve bytes, otherwise

Re: [DISCUSS] KIP-951: Leader discovery optimisations for the client

Hi Mayank, Thanks for the KIP! Questions: 1. From the standpoint of the client, does it matter if the cluster is running in KRaft mode vs. Zookeeper? Will the behavior somehow be subtlety different given that metadata propagation is handled differently between the two? 2. Is there anything

Re: [DISCUSS] KIP-951: Leader discovery optimisations for the client

Jose, Using the term "node" makes sense since it follows convention, and then "NodeEndpoints" can be reused in future. Update the KIP with protocol message changes. On Fri, Jul 14, 2023 at 5:53 PM José Armando García Sancio wrote: > Hi Mayank, > > On Thu, Jul 13, 2023 at 10:03 AM Mayank

Re: [DISCUSS] KIP-951: Leader discovery optimisations for the client

Hi Mayank/José, > On Jul 13, 2023, at 8:20 AM, José Armando García Sancio > wrote: > > Hi Mayank, thanks for the KIP. I look forward to this improvement for > new clients. > > Some comments below. > > On Thu, Jul 13, 2023 at 7:15 AM Mayank Shekhar Narula > wrote: >> Following KIP is up for

Re: [DISCUSS] KIP-951: Leader discovery optimisations for the client

Hi Ismael Right now, effort is to get baseline Vs KIP-proposed changes #s. After that will look at the alternative #s. On Fri, Jul 14, 2023 at 12:38 AM Ismael Juma wrote: > Hi Mayank, > > See my answer below. > > On Thu, Jul 13, 2023 at 10:24 AM Mayank Shekhar Narula < >

Re: [DISCUSS] KIP-951: Leader discovery optimisations for the client

Hi Mayank, On Thu, Jul 13, 2023 at 10:03 AM Mayank Shekhar Narula wrote: > 3. If I understood this correctly, certain replicas "aren't" brokers, what > are they then? In a Kafka KRaft cluster they can be either brokers, controllers or both. The term we use is node. A Kafka node can be either a

[jira] [Created] (KAFKA-15191) Add support for Micrometer Observation

Marcin Grzejszczak created KAFKA-15191: -- Summary: Add support for Micrometer Observation Key: KAFKA-15191 URL: https://issues.apache.org/jira/browse/KAFKA-15191 Project: Kafka Issue

[jira] [Resolved] (KAFKA-14462) New Group Coordinator State Machine

[ https://issues.apache.org/jira/browse/KAFKA-14462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Jacot resolved KAFKA-14462. - Fix Version/s: 3.6.0 Resolution: Fixed > New Group Coordinator State Machine >

Re: [VOTE] 3.5.1 RC0

Thank you both for testing this. Fede, that's a good catch. My bad on not catching it earlier during release prep. I will create a new RC containing the fix - https://github.com/apache/kafka/pull/14016 -- Divij Vaidya On Fri, Jul 14, 2023 at 9:29 AM Jakub Scholz wrote: > +1 (non-binding). I

Re: [DISCUSS] KIP-949: Add flag to enable the usage of topic separator in MM2 DefaultReplicationPolicy

Hi Omnia and Chris, I agree with setting "replication.policy.internal.topic.separator.enabled=true" by default, but I was wondering if we should also deprecate and remove in Kafka 4. If there is agreement in having the same behavior for internal and non-internal topics, then it should be fine, and

Re: [DISCUSS] KIP-949: Add flag to enable the usage of topic separator in MM2 DefaultReplicationPolicy

Hi Chris, I added a section for backport plan here https://cwiki.apache.org/confluence/display/KAFKA/KIP-949%3A+Add+flag+to+enable+the+usage+of+topic+separator+in+MM2+DefaultReplicationPolicy#KIP949:AddflagtoenabletheusageoftopicseparatorinMM2DefaultReplicationPolicy-Backportingplan Cheers,

Re: [DISCUSS] KIP-852 Optimize calculation of size for log in remote tier

Thank you for spotting that Luke. I have fixed the snippet now. *Satish*, I am waiting for your review on this one since you provided some comments earlier in this discussion. Please check the KIP once again when you get a chance and vote at

[jira] [Created] (KAFKA-15190) Allow configuring a streams process ID

Joe Wreschnig created KAFKA-15190: - Summary: Allow configuring a streams process ID Key: KAFKA-15190 URL: https://issues.apache.org/jira/browse/KAFKA-15190 Project: Kafka Issue Type: Wish

[jira] [Created] (KAFKA-15189) Do not initialize RemoteStorage related metrics when disabled at cluster

Divij Vaidya created KAFKA-15189: Summary: Do not initialize RemoteStorage related metrics when disabled at cluster Key: KAFKA-15189 URL: https://issues.apache.org/jira/browse/KAFKA-15189 Project:

Re: [VOTE] KIP-944 Support async runtimes in consumer, votes needed!

Hi Colin, The way I understood Philp's message is that KIP-944 also plays nice with KIP-945. But I might be mistaken. Regardless, KIP-945 does /not/ resolve the underlying problem (the need for nested consumer invocations) because it has the explicit goal of not changing the user facing

RE: Testing FixedKeyProcessor implementation using unit tests

Hi Matthias, Here's the repro of the project that has these issues https://github.com/zveljkovic/kafka-repro. Please look at the: Topology definition: https://github.com/zveljkovic/kafka-repro/blob/master/src/main/java/com/example/demo/DemoApplication.java FixedKeyProcessor:

Re: [VOTE] 3.5.1 RC0

+1 (non-binding). I used the staged Scala 2.13 binaries and the Maven artifacts for my tests and all seems to work fine. Thanks. Jakub On Wed, Jul 12, 2023 at 12:03 PM Divij Vaidya wrote: > Hello Kafka users, developers and client-developers, > > This is the first candidate for release of

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #1999

See Changes: -- [...truncated 379456 lines...] > Task :connect:api:compileTestJava UP-TO-DATE > Task :connect:api:testClasses UP-TO-DATE > Task