Re: [VOTE] KIP-919: Allow AdminClient to Talk Directly with the KRaft Controller Quorum and add Controller Registration

Hi all, With binding +1s from Ron, David, Divij, ziming, and Luke, the vote passes. Thanks, everyone. Best, Colin On Wed, Jul 26, 2023, at 16:02, Colin McCabe wrote: > On Wed, Jul 26, 2023, at 07:09, Ron Dagostino wrote: >> Thanks, Colin. +1 (binding) from me. >> >> I will note that Ziming

Re: Re: [DISCUSS] KIP-951: Leader discovery optimisations for the client

Thanks for adding the benchmark results, Crispin! IMO, 2~5% performance improvement is small, but given the change is small, cost is also small (only append endpoint info when NOT_LEADER_OR_FOLLOWER.. etc error), I think it is worth doing it. Thank you. Luke On Wed, Jul 26, 2023 at 12:33 AM

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #2045

See

Re: [DISCUSS] KIP-759: Unneeded repartition canceling

Very good catch, Matthias. I updated the KIP to state that the new DSLOperation will return a new, mutated KStream. Thank you, Shay On Wed, Jul 26, 2023 at 6:13 PM Matthias J. Sax wrote: > One last question. What should happen for the following case: > > KStream myStream =

Build failed in Jenkins: Kafka » Kafka Branch Builder » 3.5 #50

See Changes: -- [...truncated 467493 lines...] [Pipeline] // node [Pipeline] } [Pipeline] // timestamps [Pipeline] } [Pipeline] // timeout [Pipeline] } [Pipeline] // stage

Re: [DISCUSS] KIP-759: Unneeded repartition canceling

One last question. What should happen for the following case: KStream myStream = build.stream(...).map(...); myStream.markAsPartiitoned().groupByKey().aggregate(...); myStream.join(...) The question is about the "fan-out" pattern. `myStream`, which is marked for partitioning, is fed into two

Jenkins build is unstable: Kafka » Kafka Branch Builder » trunk #2044

See

Re: [DISCUSS] KIP-954: expand default DSL store configuration to custom types

Thanks for the feedback Bruno -- sounds like we're getting close to a final consensus here. It sounds like the two main (only?) semi-unresolved questions that still have differing opinions floating around are whether to deprecate the old config, and what to name the new config + interface.

Re: [DISCUSS] KIP-759: Unneeded repartition canceling

Hi John, Thanks for your reply. I updated the KIP to reflect the changes we discussed in the thread today. #1 is duly noted, I learned from the examples Sophie sent earlier! =) In the new version, I also talked about why IQ and joins will not work with the interface and talked about the

Re: [VOTE] KIP-919: Allow AdminClient to Talk Directly with the KRaft Controller Quorum and add Controller Registration

On Wed, Jul 26, 2023, at 07:09, Ron Dagostino wrote: > Thanks, Colin. +1 (binding) from me. > > I will note that Ziming mentioned in the DISCUSS thread that "There is > a mistake that we use `--bootstrap-server` instead of > `--bootstrap-server(s)`, so should we also change the new argument >

Re: [DISCUSS] KIP-960: Support interactive queries (IQv2) for versioned state stores

Thanks for the KIP Alieh. Glad to see that we can add IQ to the new versioned stores! Couple of questions: single-key lookup with timestamp (upper) bound Not sure if "bound" is the right term? In the end, it's a point lookup for a key plus timestamps, so it's an as-of timestamp (not a

Re: [DISCUSS] KIP-759: Unneeded repartition canceling

Hello Shay, Thanks for the KIP! I just took a look in preparation to vote, and there are two small-ish things that I'd like to fix first. Apologies if this stuff has already come up in the discussion thread; I only skimmed it. 1. The KIP only mentions the name of the method instead of

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #2043

See Changes: -- [...truncated 392632 lines...] Gradle Test Run :streams:integrationTest > Gradle Test Executor 184 > TableTableJoinIntegrationTest > [caching enabled =

Re: Streams: Global Store Processors

That's correct. And yes, it's a public contract and thus a KIP would be needed to change (or remove it). Deprecation implies that the API is kept for at least 3 releases (ie, one year), plus it can only be removed in a major release. For example, if we deprecated something in 2.1 release, we

[jira] [Resolved] (KAFKA-15235) No test coverage reports for Java due to settings for Jacoco being incompatible with Gradle 8.x

[ https://issues.apache.org/jira/browse/KAFKA-15235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya resolved KAFKA-15235. -- Resolution: Fixed > No test coverage reports for Java due to settings for Jacoco being >

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #2042

See Changes: -- [...truncated 390703 lines...] Gradle Test Run :streams:integrationTest > Gradle Test Executor 184 > RestoreIntegrationTest >

Streams: Global Store Processors

Hi all, In this JIRA ticket: https://issues.apache.org/jira/browse/KAFKA-7663 it's documented that Global Stores are bypassed on restoration. Consequently, the input topic to a Global Store needs to essentially be a changelog topic as the keys and values are copied directly into the store. I

Re: Apache Kafka 3.6.0 release

Yes, still need one more binding vote to pass. I'll send a reminder if the vote is still pending after the waiting period. Cheers, From: dev@kafka.apache.org At: 07/26/23 12:17:10 UTC-4:00To: dev@kafka.apache.org Subject: Re: Apache Kafka 3.6.0 release Hi Hector/Yash, Are you planning to

Re: Apache Kafka 3.6.0 release

Sorry, my bad (you can tell this is the first time one of my KIPs have made it this far :)) From: dev@kafka.apache.org At: 07/26/23 10:38:21 UTC-4:00To: dev@kafka.apache.org Subject: Re: Apache Kafka 3.6.0 release Hi Hector, KIP-959 actually still requires 2 more binding votes to be accepted

Re: [VOTE] KIP-959 Add BooleanConverter to Kafka Connect

Hey Hector, Thanks for the straightforward and clear KIP! +1 (binding) Thanks, Greg On Wed, Jul 26, 2023 at 5:16 AM Chris Egerton wrote: > > +1 (binding) > > Thanks Hector! > > On Wed, Jul 26, 2023 at 3:18 AM Kamal Chandraprakash < > kamal.chandraprak...@gmail.com> wrote: > > > +1

Re: Apache Kafka 3.6.0 release

Hi Hector/Yash, Are you planning to reach out to other committers to vote on the KIP and close the vote in the next couple of days? Thanks, Satish. On Wed, 26 Jul 2023 at 20:08, Yash Mayya wrote: > > Hi Hector, > > KIP-959 actually still requires 2 more binding votes to be accepted ( >

[jira] [Created] (KAFKA-15259) Kafka Streams does not continue processing due to rollback despite ProductionExceptionHandlerResponse.CONTINUE if using execute_once

Tomonari Yamashita created KAFKA-15259: -- Summary: Kafka Streams does not continue processing due to rollback despite ProductionExceptionHandlerResponse.CONTINUE if using execute_once Key: KAFKA-15259 URL:

[VOTE] KIP-954: expand default DSL store configuration to custom types

Hello Everyone, Opening the voting for KIP-954. The discussion is converging, but please feel free to chime in on the last few conversation points if you aren't happy with where it settled.

Re: [VOTE] KIP-951: Leader discovery optimisations for the client

+1 (non-binding) Thanks Mayank! On Mon, Jul 24, 2023, at 11:24 AM, Mayank Shekhar Narula wrote: > Hi all > > Opening the voting process for this KIP, given the deadline for KIP > acceptance for 3.6 is impending. Open to keep the discussion going if the > community needs more time to converge. >

Re: [DISCUSS] KIP-954: expand default DSL store configuration to custom types

Thanks for the comments Bruno! A3. Oops... I think I didn't do a great job updating the KIP to reflect Guozhang's suggestion. This seems like the last point of contention, where we have two options: 1. Deprecate the config entirely and replace IN_MEMORY/ROCKSDB with implementations of the

Jenkins build is unstable: Kafka » Kafka Branch Builder » trunk #2041

See

Re: Apache Kafka 3.6.0 release

Hi Hector, KIP-959 actually still requires 2 more binding votes to be accepted ( https://cwiki.apache.org/confluence/display/KAFKA/Bylaws#Bylaws-Approvals). The non-binding votes from people who aren't committers (including myself) don't count towards the required lazy majority. Thanks, Yash On

[DISCUSS] KIP-960: Support interactive queries (IQv2) for versioned state stores

Hi all, I would like to propose a KIP to support IQv2 for versioned state stores. https://cwiki.apache.org/confluence/display/KAFKA/KIP-960%3A+Support+interactive+queries+%28IQv2%29+for+versioned+state+stores Looking forward to your feedback! Cheers, Alieh

Re: [DISCUSS] KIP-955: Add stream-table join on foreign key

Hello Matthias, Thank you for this response. It provides the context for a good discussion related to the need for this new interface. The use case I have in mind is not really a stream enrichment which usually implies that the event has a primary key to some external info and that external info

Re: [VOTE] KIP-919: Allow AdminClient to Talk Directly with the KRaft Controller Quorum and add Controller Registration

Thanks, Colin. +1 (binding) from me. I will note that Ziming mentioned in the DISCUSS thread that "There is a mistake that we use `--bootstrap-server` instead of `--bootstrap-server(s)`, so should we also change the new argument `--bootstrap-controller` (no s).". I agree that this is an

Re: [VOTE] KIP-919: Allow AdminClient to Talk Directly with the KRaft Controller Quorum and add Controller Registration

Thanks for driving this KIP, Colin! +1 binding -David On Wed, Jul 26, 2023 at 8:58 AM Divij Vaidya wrote: > +1 (binding) > > -- > Divij Vaidya > > > On Wed, Jul 26, 2023 at 2:56 PM ziming deng > wrote: > > > > +1 (binding) from me. > > > > Thanks for the KIP! > > > > -- > > Ziming > > > > >

Re: Apache Kafka 3.6.0 release

Hi Hector, Thanks for the update on KIP-959. ~Satish. On Wed, 26 Jul 2023 at 18:38, Hector Geraldino (BLOOMBERG/ 919 3RD A) wrote: > > Hi Satish, > > I added KIP-959 [1] to the list. The KIP has received enough votes to pass, > but I'm waiting the 72 hours before announcing the results.

[jira] [Resolved] (KAFKA-15243) User creation mismatch

[ https://issues.apache.org/jira/browse/KAFKA-15243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikumar resolved KAFKA-15243. --- Fix Version/s: 3.6.0 Resolution: Fixed > User creation mismatch > -- > >

Re: [DISCUSS] KIP-954: expand default DSL store configuration to custom types

Hi, Sorry for being late to the party! A1: I agree with Sophie, Guozhang, and Almog not to block the KIP on gaps in the implementation. A2: I am happy with not considering anything special w.r.t. versioned state stores in this KIP. A3: Here I agree with Sophie to deprecate the old config.

Re: Apache Kafka 3.6.0 release

Hi Satish, I added KIP-959 [1] to the list. The KIP has received enough votes to pass, but I'm waiting the 72 hours before announcing the results. There's also a (small) PR with the implementation for this KIP that hopefully will get reviewed/merged soon. Best, [1]

Re: [VOTE] KIP-919: Allow AdminClient to Talk Directly with the KRaft Controller Quorum and add Controller Registration

+1 (binding) -- Divij Vaidya On Wed, Jul 26, 2023 at 2:56 PM ziming deng wrote: > > +1 (binding) from me. > > Thanks for the KIP! > > -- > Ziming > > > On Jul 26, 2023, at 20:18, Luke Chen wrote: > > > > +1 (binding) from me. > > > > Thanks for the KIP! > > > > Luke > > > > On Tue, Jul 25,

Re: [VOTE] KIP-919: Allow AdminClient to Talk Directly with the KRaft Controller Quorum and add Controller Registration

+1 (binding) from me. Thanks for the KIP! -- Ziming > On Jul 26, 2023, at 20:18, Luke Chen wrote: > > +1 (binding) from me. > > Thanks for the KIP! > > Luke > > On Tue, Jul 25, 2023 at 1:24 AM Colin McCabe wrote: > >> Hi all, >> >> I'd like to start the vote for KIP-919: Allow

Re: [VOTE] KIP-919: Allow AdminClient to Talk Directly with the KRaft Controller Quorum and add Controller Registration

+1 (binding) from me. Thanks for the KIP! Luke On Tue, Jul 25, 2023 at 1:24 AM Colin McCabe wrote: > Hi all, > > I'd like to start the vote for KIP-919: Allow AdminClient to Talk Directly > with the KRaft Controller Quorum and add Controller Registration. > > The KIP is here:

Re: [ANNOUNCE] Apache Kafka 3.5.1

Thanks, Divij! On Wed, Jul 26, 2023 at 3:40 AM Bruno Cadonna wrote: > Thanks Divij! > > On 7/24/23 5:22 PM, Federico Valeri wrote: > > Thanks Divij :) > > > > On Mon, Jul 24, 2023 at 2:28 PM Luke Chen wrote: > >> > >> Thanks for running the release! > >> > >> Luke > >> > >> On Mon, Jul 24,

Re: [VOTE] KIP-959 Add BooleanConverter to Kafka Connect

+1 (binding) Thanks Hector! On Wed, Jul 26, 2023 at 3:18 AM Kamal Chandraprakash < kamal.chandraprak...@gmail.com> wrote: > +1 (non-binding). Thanks for the KIP! > > On Tue, Jul 25, 2023 at 11:12 PM Yash Mayya wrote: > > > Hi Hector, > > > > Thanks for the KIP! > > > > +1 (non-binding) > > > >

[jira] [Created] (KAFKA-15258) Consider moving MockAdminClient to the public API

Mickael Maison created KAFKA-15258: -- Summary: Consider moving MockAdminClient to the public API Key: KAFKA-15258 URL: https://issues.apache.org/jira/browse/KAFKA-15258 Project: Kafka Issue

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #2040

See Changes: -- [...truncated 392190 lines...] Gradle Test Run :streams:integrationTest > Gradle Test Executor 177 > RestoreIntegrationTest >

[jira] [Created] (KAFKA-15257) Support interactive queries (IQv2) with versioned state store

Alieh Saeedi created KAFKA-15257: Summary: Support interactive queries (IQv2) with versioned state store Key: KAFKA-15257 URL: https://issues.apache.org/jira/browse/KAFKA-15257 Project: Kafka

[jira] [Created] (KAFKA-15256) Add code reviewers to contributors list in release email

Divij Vaidya created KAFKA-15256: Summary: Add code reviewers to contributors list in release email Key: KAFKA-15256 URL: https://issues.apache.org/jira/browse/KAFKA-15256 Project: Kafka

Re: Debugging Jenkins test failures

Hi Kirk I have been using this new tool to analyze the trends of test failures: https://ge.apache.org/scans/tests?search.relativeStartTime=P28D=kafka=Europe/Berlin and general build failures: https://ge.apache.org/scans/failures?search.relativeStartTime=P28D=kafka=Europe/Berlin About the

[jira] [Reopened] (KAFKA-14908) Sporadic "Address already in use" when starting kafka cluster embedded within tests

[ https://issues.apache.org/jira/browse/KAFKA-14908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Divij Vaidya reopened KAFKA-14908: -- > Sporadic "Address already in use" when starting kafka cluster embedded within > tests >

[REVIEW REQUEST] AdminUtils rewritten in java

Hello. I’m working on [1]. The goal of ticket is to rewire `ReassignPartitionCommand` in java. The command itself are relatively complex and has several dependencies on the core module. The main is `AdminUtils`. `AdminUtils` seems to me important part of Kafka logic itself so I rewrite it in

Re: [ANNOUNCE] Apache Kafka 3.5.1

Thanks Divij! On 7/24/23 5:22 PM, Federico Valeri wrote: Thanks Divij :) On Mon, Jul 24, 2023 at 2:28 PM Luke Chen wrote: Thanks for running the release! Luke On Mon, Jul 24, 2023 at 6:35 PM Manyanda Chitimbo < manyanda.chiti...@gmail.com> wrote: Thanks for running the release Divij.

Re: Request permission to contribute

Hi Taras, I set you up for the Apache Kafka wiki. Let me know if you still miss permissions. Thank you for your interest in Apache Kafka! Best, Bruno On 7/25/23 9:49 AM, Taras Ledkov wrote: Hi Guozhang, Thanks for your attention. I'm a contributor of other Apache project (Ignite). Now I

Re: [VOTE] KIP-959 Add BooleanConverter to Kafka Connect

+1 (non-binding). Thanks for the KIP! On Tue, Jul 25, 2023 at 11:12 PM Yash Mayya wrote: > Hi Hector, > > Thanks for the KIP! > > +1 (non-binding) > > Thanks, > Yash > > On Tue, Jul 25, 2023 at 11:01 PM Andrew Schofield < > andrew_schofield_j...@outlook.com> wrote: > > > Thanks for the KIP. As

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #2039

See Changes: -- [...truncated 294679 lines...] Gradle Test Run :streams:integrationTest > Gradle Test Executor 181 > TableTableJoinIntegrationTest > [caching enabled =

[VOTE] KIP-942: Add Power(ppc64le) support

I'd like to call a vote on KIP-942

[jira] [Resolved] (KAFKA-15255) Task is not stopped until the poll interval passes in case of task restarting.

[ https://issues.apache.org/jira/browse/KAFKA-15255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikita resolved KAFKA-15255. Resolution: Duplicate > Task is not stopped until the poll interval passes in case of task restarting. >

[jira] [Resolved] (KAFKA-15254) Task is not stopped until the poll interval passes in case of task restarting.

[ https://issues.apache.org/jira/browse/KAFKA-15254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikita resolved KAFKA-15254. Resolution: Duplicate > Task is not stopped until the poll interval passes in case of task restarting. >

[jira] [Created] (KAFKA-15255) Task is not stopped until the poll interval passes in case of task restarting.

Nikita created KAFKA-15255: -- Summary: Task is not stopped until the poll interval passes in case of task restarting. Key: KAFKA-15255 URL: https://issues.apache.org/jira/browse/KAFKA-15255 Project: Kafka

[jira] [Resolved] (KAFKA-15253) Task is not stopped until the poll interval passes in case of task restarting.

[ https://issues.apache.org/jira/browse/KAFKA-15253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikita resolved KAFKA-15253. Resolution: Duplicate > Task is not stopped until the poll interval passes in case of task restarting. >

[jira] [Created] (KAFKA-15254) Task is not stopped until the poll interval passes in case of task restarting.

Nikita created KAFKA-15254: -- Summary: Task is not stopped until the poll interval passes in case of task restarting. Key: KAFKA-15254 URL: https://issues.apache.org/jira/browse/KAFKA-15254 Project: Kafka

[jira] [Created] (KAFKA-15253) Task is not stopped until the poll interval passes in case of task restarting.

Nikita created KAFKA-15253: -- Summary: Task is not stopped until the poll interval passes in case of task restarting. Key: KAFKA-15253 URL: https://issues.apache.org/jira/browse/KAFKA-15253 Project: Kafka

[jira] [Created] (KAFKA-15252) Task is not stopped until the poll interval passes in case of task restarting.

Nikita created KAFKA-15252: -- Summary: Task is not stopped until the poll interval passes in case of task restarting. Key: KAFKA-15252 URL: https://issues.apache.org/jira/browse/KAFKA-15252 Project: Kafka