Re: Apache Kafka 3.7.0 Release

2024-02-12 Thread Mayank Shekhar Narula
e fact that it's for high partition counts (a > definition here would help though) > > Reminder - RC4 is out for vote right now. > > Best, > Stanislav > > On Tue, Feb 6, 2024 at 5:25 PM Mayank Shekhar Narula < > mayanks.nar...@gmail.com> wrote: > > > Hi Fol

Re: [VOTE] KIP-951: Leader discovery optimisations for the client

2024-02-06 Thread Mayank Shekhar Narula
as a blocker for the release. > > Best, > David > > On Tue, Feb 6, 2024 at 5:09 PM Mayank Shekhar Narula < > mayanks.nar...@gmail.com> wrote: > > > Hi Folks > > > > KIP-951 was delivered fully in AK 3.7. Its 1st optimisation was delivered > > in 3.6.1,

Re: Apache Kafka 3.7.0 Release

2024-02-06 Thread Mayank Shekhar Narula
> >> >> Thanks for the response. The only reason for asking the > > > > question of > > > > >>>> >> >> publishing the metadata is because that's present in > > previous > > > > client > > > > >>>> >> >> releases. For more context, the description of PR > > > > >>>> >> >> <https://github.com/apache/kafka/pull/15127> holds the > > > details > > > > and > > > > >>>> >> waiting > > > > >>>> >> >> for the confirmation there prior to the merge. > > > > >>>> >> >> > > > > >>>> >> >> Regards, > > > > >>>> >> >> Apoorv Mittal > > > > >>>> >> >> +44 7721681581 > > > > >>>> >> >> > > > > >>>> >> >> > > > > >>>> >> >> On Fri, Jan 5, 2024 at 10:22 PM Colin McCabe < > > > > cmcc...@apache.org> > > > > >>>> >> wrote: > > > > >>>> >> >> > > > > >>>> >> >>> metadata is an internal gradle module. It is not used by > > > > clients. > > > > >>>> So I > > > > >>>> >> >>> don't see why you would want to publish it (unless I'm > > > > >>>> misunderstanding > > > > >>>> >> >>> something). > > > > >>>> >> >>> > > > > >>>> >> >>> best, > > > > >>>> >> >>> Colin > > > > >>>> >> >>> > > > > >>>> >> >>> > > > > >>>> >> >>> On Fri, Jan 5, 2024, at 10:05, Stanislav Kozlovski wrote: > > > > >>>> >> >>> > Thanks for reporting the blockers, folks. Good job > > finding. > > > > >>>> >> >>> > > > > > >>>> >> >>> > I have one ask - can anybody with Gradle expertise help > > > > review > > > > >>>> this > > > > >>>> >> small > > > > >>>> >> >>> > PR? https://github.com/apache/kafka/pull/15127 (+1, > -1) > > > > >>>> >> >>> > In particular, we are wondering whether we need to > > publish > > > > module > > > > >>>> >> >>> metadata > > > > >>>> >> >>> > as part of the gradle publishing process. > > > > >>>> >> >>> > > > > > >>>> >> >>> > > > > > >>>> >> >>> > On Fri, Jan 5, 2024 at 3:56 PM Proven Provenzano > > > > >>>> >> >>> > wrote: > > > > >>>> >> >>> > > > > > >>>> >> >>> >> We have potentially one more blocker > > > > >>>> >> >>> >> https://issues.apache.org/jira/browse/KAFKA-16082 > which > > > > might > > > > >>>> >> cause a > > > > >>>> >> >>> data > > > > >>>> >> >>> >> loss scenario with JBOD in KRaft. > > > > >>>> >> >>> >> Initial analysis thought this is a problem and further > > > > review > > > > >>>> looks > > > > >>>> >> >>> like it > > > > >>>> >> >>> >> isn't but we are continuing to dig into the issue to > > > ensure > > > > that > > > > >>>> it > > > > >>>> >> >>> isn't. > > > > >>>> >> >>> >> We would request feedback on the bug from anyone who > is > > > > familiar > > > > >>>> >> with > > > > >>>> >> >>> this > > > > >>>> >> >>> >> code. > > > > >>>> >> >>> >> > > > > >>>> >> >>> >> --Proven > > > > >>>> >> >>> >> > > > > >>>> >> >>> > > > > > >>>> >> >>> > > > > > >>>> >> >>> > -- > > > > >>>> >> >>> > Best, > > > > >>>> >> >>> > Stanislav > > > > >>>> >> >>> > > > > >>>> >> > > > > >>>> > > > > > > > > > > > > > -- > > > Best, > > > Stanislav > > > > > > -- Regards, Mayank Shekhar Narula

Re: [VOTE] KIP-951: Leader discovery optimisations for the client

2024-02-06 Thread Mayank Shekhar Narula
to Cluster.java, and a public constructor to PartitionInfo.java. Let me know your thoughts. On Wed, Oct 4, 2023 at 10:09 AM Mayank Shekhar Narula < mayanks.nar...@gmail.com> wrote: > Summarising, there are 5 binding votes(Luke, Jose, Jun, David, Jason), and > 1 non-bindi

[jira] [Created] (KAFKA-16226) Performance regression in Trogdor benchmark with high partition counts

2024-02-06 Thread Mayank Shekhar Narula (Jira)
Mayank Shekhar Narula created KAFKA-16226: - Summary: Performance regression in Trogdor benchmark with high partition counts Key: KAFKA-16226 URL: https://issues.apache.org/jira/browse/KAFKA-16226

[jira] [Resolved] (KAFKA-15970) KIP-951, port newly added tests in FetcherTest.java to FetchRequestManagerTest.ajva

2024-02-06 Thread Mayank Shekhar Narula (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mayank Shekhar Narula resolved KAFKA-15970. --- Resolution: Fixed > KIP-951, port newly added tests in FetcherTest.j

[jira] [Created] (KAFKA-15970) KIP-951, port newly added tests in FetcherTest.java to FetchRequestManagerTest.ajva

2023-12-04 Thread Mayank Shekhar Narula (Jira)
Mayank Shekhar Narula created KAFKA-15970: - Summary: KIP-951, port newly added tests in FetcherTest.java to FetchRequestManagerTest.ajva Key: KAFKA-15970 URL: https://issues.apache.org/jira/browse/KAFKA

Re: Apache Kafka 3.7.0 Release

2023-11-21 Thread Mayank Shekhar Narula
> > > > > > > >> > > > > > > > > > > >> > > *KIP Freeze - 18th November *(Saturday) > > > > > > > > >> > > *This is 1 month and four days from now - rather > short - > > > but > > > > > I'm > > > > > > > > >> afraid > > > > > > > > >> > is > > > > > > > > >> > > the only lever that's easy to pull forward.* > > > > > > > > >> > > As usual, a KIP must be accepted by this date in order > > to > > > be > > > > > > > > >> considered > > > > > > > > >> > for > > > > > > > > >> > > this release. Note, any KIP that may not be > implemented > > > in a > > > > > > week, > > > > > > > > or > > > > > > > > >> > that > > > > > > > > >> > > might destabilize the release, should be deferred. > > > > > > > > >> > > > > > > > > > > >> > > *Feature Freeze - 8th December* (Friday) > > > > > > > > >> > > *This follows 3 weeks after the KIP Freeze, as has > been > > > the > > > > > case > > > > > > > in > > > > > > > > >> our > > > > > > > > >> > > latest releases.* > > > > > > > > >> > > By this point, we want all major features to be > merged & > > > us > > > > to > > > > > > be > > > > > > > > >> working > > > > > > > > >> > > on stabilisation. Minor features should have PRs, the > > > > release > > > > > > > branch > > > > > > > > >> > should > > > > > > > > >> > > be cut; anything not in this state will be > automatically > > > > moved > > > > > > to > > > > > > > > the > > > > > > > > >> > next > > > > > > > > >> > > release in JIRA > > > > > > > > >> > > > > > > > > > > >> > > *Code Freeze - 20th December* (Wednesday) > > > > > > > > >> > > > > > > > > > > >> > > *Critically, this is before the holiday season and > ends > > in > > > > the > > > > > > > > middle > > > > > > > > >> of > > > > > > > > >> > > the week, to give contributors more time and > flexibility > > > to > > > > > > > address > > > > > > > > >> any > > > > > > > > >> > > last-minute without eating into the time people > usually > > > take > > > > > > > > >> holidays. It > > > > > > > > >> > > comes 12 days after the Feature Freeze.This is two > days > > > > > shorter > > > > > > > than > > > > > > > > >> the > > > > > > > > >> > > usual code freeze window. I don't have a strong > opinion > > > and > > > > am > > > > > > > open > > > > > > > > to > > > > > > > > >> > > extend it to Friday, or trade off a day/two with the > > > KF<->FF > > > > > > date > > > > > > > > >> range.* > > > > > > > > >> > > > > > > > > > > >> > > *Release -* *after January 3rd*. > > > > > > > > >> > > *It comes after a minimum of two weeks of > stabilization, > > > so > > > > > the > > > > > > > > >> earliest > > > > > > > > >> > we > > > > > > > > >> > > can start releasing is January 3rd. We will move as > fast > > > as > > > > we > > > > > > can > > > > > > > > and > > > > > > > > >> > aim > > > > > > > > >> > > completing it as early in January as possible.* > > > > > > > > >> > > > > > > > > > > >> > > As for the initially-populated KIPs in the release > > plan, I > > > > did > > > > > > the > > > > > > > > >> > > following: > > > > > > > > >> > > > > > > > > > > >> > > I kept 4 KIPs that were mentioned in 3.6, saying they > > > would > > > > > have > > > > > > > > minor > > > > > > > > >> > > parts finished in 3.7 (as the major ones went out in > > 3.6) > > > > > > > > >> > > - KIP-405 Tiered Storage mentioned a major part went > out > > > > with > > > > > > 3.6 > > > > > > > > and > > > > > > > > >> the > > > > > > > > >> > > remainder will come with 3.7 > > > > > > > > >> > > - KIP-890 mentioned Part 1 shipped in 3.6. I am > assuming > > > the > > > > > > > > remainder > > > > > > > > >> > will > > > > > > > > >> > > come in 3.7, and have contacted the author to confirm. > > > > > > > > >> > > - KIP-926 was partially implemented in 3.6. I am > > assuming > > > > the > > > > > > > > >> remainder > > > > > > > > >> > > will come in 3.7, and have contacted the author to > > > confirm. > > > > > > > > >> > > - KIP-938 mentioned that the majority was completed > and > > a > > > > > small > > > > > > > > >> remainder > > > > > > > > >> > > re: ForwardingManager metrics will come in 3.7. I have > > > > > contacted > > > > > > > the > > > > > > > > >> > author > > > > > > > > >> > > to confirm. > > > > > > > > >> > > > > > > > > > > >> > > I then went through the JIRA filter which looks at > open > > > > issues > > > > > > > with > > > > > > > > a > > > > > > > > >> Fix > > > > > > > > >> > > Version of 3.7 and added KIP-770, KIP-858, and > KIP-980. > > > > > > > > >> > > I also found a fair amount of JIRAs that were > targeting > > > the > > > > > 3.7 > > > > > > > > >> release > > > > > > > > >> > but > > > > > > > > >> > > consecutively had no activity on them for the past few > > > > > releases. > > > > > > > For > > > > > > > > >> most > > > > > > > > >> > > of those, I pinged the author and explicitly asked if > > it's > > > > > going > > > > > > > to > > > > > > > > >> aim > > > > > > > > >> > to > > > > > > > > >> > > make it to 3.7. I have not included those here and > will > > > not > > > > > > until > > > > > > > I > > > > > > > > >> hear > > > > > > > > >> > > confirmation. > > > > > > > > >> > > > > > > > > > > >> > > Please review the plan and provide any additional > > > > information > > > > > or > > > > > > > > >> updates > > > > > > > > >> > > regarding KIPs that target this release version (3.7). > > > > > > > > >> > > If you have authored any KIPs that have an inaccurate > > > status > > > > > in > > > > > > > the > > > > > > > > >> list, > > > > > > > > >> > > or are not in the list and should be, or are in the > list > > > and > > > > > > > should > > > > > > > > >> not > > > > > > > > >> > be > > > > > > > > >> > > - please inform me in this thread so that I can keep > the > > > > > > document > > > > > > > > >> > accurate > > > > > > > > >> > > and up to date. > > > > > > > > >> > > > > > > > > > > >> > > Excited to get this release going! > > > > > > > > >> > > > > > > > > > > >> > > All the best, > > > > > > > > >> > > Stanislav > > > > > > > > >> > > > > > > > > > > >> > > On Tue, Oct 10, 2023 at 9:12 AM Bruno Cadonna < > > > > > > cado...@apache.org > > > > > > > > > > > > > > > > >> > wrote: > > > > > > > > >> > > > > > > > > > > >> > > > Thanks Stan! > > > > > > > > >> > > > > > > > > > > > >> > > > +1 > > > > > > > > >> > > > > > > > > > > > >> > > > Best, > > > > > > > > >> > > > Bruno > > > > > > > > >> > > > > > > > > > > > >> > > > On 10/10/23 7:24 AM, Luke Chen wrote: > > > > > > > > >> > > > > Thanks Stanislav! > > > > > > > > >> > > > > > > > > > > > > >> > > > > On Tue, Oct 10, 2023 at 3:05 AM Josep Prat > > > > > > > > >> > > > > > > > > >> > > > > > > > > > > > >> > > > > wrote: > > > > > > > > >> > > > > > > > > > > > > >> > > > >> Thanks Stanislav! > > > > > > > > >> > > > >> > > > > > > > > >> > > > >> ——— > > > > > > > > >> > > > >> Josep Prat > > > > > > > > >> > > > >> > > > > > > > > >> > > > >> Aiven Deutschland GmbH > > > > > > > > >> > > > >> > > > > > > > > >> > > > >> Alexanderufer 3-7, 10117 Berlin > > > > > > > > >> > > > >> > > > > > > > > >> > > > >> Amtsgericht Charlottenburg, HRB 209739 B > > > > > > > > >> > > > >> > > > > > > > > >> > > > >> Geschäftsführer: Oskari Saarenmaa & Hannu > Valtonen > > > > > > > > >> > > > >> > > > > > > > > >> > > > >> m: +491715557497 > > > > > > > > >> > > > >> > > > > > > > > >> > > > >> w: aiven.io > > > > > > > > >> > > > >> > > > > > > > > >> > > > >> e: josep.p...@aiven.io > > > > > > > > >> > > > >> > > > > > > > > >> > > > >> On Mon, Oct 9, 2023, 20:05 Chris Egerton < > > > > > > > > >> fearthecel...@gmail.com> > > > > > > > > >> > > > wrote: > > > > > > > > >> > > > >> > > > > > > > > >> > > > >>> +1, thanks Stanislav! > > > > > > > > >> > > > >>> > > > > > > > > >> > > > >>> On Mon, Oct 9, 2023, 14:02 Bill Bejeck < > > > > > bbej...@gmail.com > > > > > > > > > > > > > > > >> wrote: > > > > > > > > >> > > > >>> > > > > > > > > >> > > > >>>> +1 > > > > > > > > >> > > > >>>> > > > > > > > > >> > > > >>>> Thanks, Stanislav! > > > > > > > > >> > > > >>>> > > > > > > > > >> > > > >>>> -Bill > > > > > > > > >> > > > >>>> > > > > > > > > >> > > > >>>> On Mon, Oct 9, 2023 at 1:59 PM Ismael Juma < > > > > > > > > m...@ismaeljuma.com> > > > > > > > > >> > > wrote: > > > > > > > > >> > > > >>>> > > > > > > > > >> > > > >>>>> Thanks for volunteering Stanislav! > > > > > > > > >> > > > >>>>> > > > > > > > > >> > > > >>>>> Ismael > > > > > > > > >> > > > >>>>> > > > > > > > > >> > > > >>>>> On Mon, Oct 9, 2023 at 10:51 AM Stanislav > > > Kozlovski > > > > > > > > >> > > > >>>>> wrote: > > > > > > > > >> > > > >>>>> > > > > > > > > >> > > > >>>>>> Hey all! > > > > > > > > >> > > > >>>>>> > > > > > > > > >> > > > >>>>>> I would like to volunteer to be the release > > > manager > > > > > > > driving > > > > > > > > >> the > > > > > > > > >> > > > >> next > > > > > > > > >> > > > >>>>>> release - Apache Kafka *3.7.0*. > > > > > > > > >> > > > >>>>>> > > > > > > > > >> > > > >>>>>> If there are no objections, I will start and > > > share > > > > a > > > > > > > > release > > > > > > > > >> > plan > > > > > > > > >> > > > >>> soon > > > > > > > > >> > > > >>>>>> enough! > > > > > > > > >> > > > >>>>>> > > > > > > > > >> > > > >>>>>> Cheers, > > > > > > > > >> > > > >>>>>> Stanislav > > > > > > > > >> > > > >>>>>> > > > > > > > > >> > > > >>>>> > > > > > > > > >> > > > >>>> > > > > > > > > >> > > > >>> > > > > > > > > >> > > > >> > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > -- > > > > > > > > >> > > Best, > > > > > > > > >> > > Stanislav > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> -- > > > > > > > > >> Best, > > > > > > > > >> Stanislav > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > Best, > > > > > > > Stanislav > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Best, > > > > > Stanislav > > > > > > > > > > > > > > > > > -- > Best, > Stanislav > -- Regards, Mayank Shekhar Narula

[jira] [Resolved] (KAFKA-15868) KIP-951 - Leader discovery optimisations for the client

2023-11-21 Thread Mayank Shekhar Narula (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mayank Shekhar Narula resolved KAFKA-15868. --- Resolution: Fixed > KIP-951 - Leader discovery optimisations for the cli

[jira] [Created] (KAFKA-15868) KIP-951 - Leader discovery optimisations for the client

2023-11-21 Thread Mayank Shekhar Narula (Jira)
Mayank Shekhar Narula created KAFKA-15868: - Summary: KIP-951 - Leader discovery optimisations for the client Key: KAFKA-15868 URL: https://issues.apache.org/jira/browse/KAFKA-15868 Project

[jira] [Resolved] (KAFKA-15824) SubscriptionState's maybeValidatePositionForCurrentLeader should handle partition which isn't subscribed yet

2023-11-15 Thread Mayank Shekhar Narula (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mayank Shekhar Narula resolved KAFKA-15824. --- Resolution: Fixed > SubscriptionStat

[jira] [Created] (KAFKA-15824) SubscriptionState's maybeValidatePositionForCurrentLeader should handle partition which isn't subscribed yet

2023-11-14 Thread Mayank Shekhar Narula (Jira)
Mayank Shekhar Narula created KAFKA-15824: - Summary: SubscriptionState's maybeValidatePositionForCurrentLeader should handle partition which isn't subscribed yet Key: KAFKA-15824 URL: https

[jira] [Resolved] (KAFKA-15415) In Java-client, backoff should be skipped for retried producer-batch to a new leader

2023-10-27 Thread Mayank Shekhar Narula (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-15415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mayank Shekhar Narula resolved KAFKA-15415. --- Resolution: Fixed > In Java-client, backoff should be skipped for retr

[jira] [Created] (KAFKA-15627) KIP-951, Java client changes to incorporate the leader discovery optimisations

2023-10-17 Thread Mayank Shekhar Narula (Jira)
Mayank Shekhar Narula created KAFKA-15627: - Summary: KIP-951, Java client changes to incorporate the leader discovery optimisations Key: KAFKA-15627 URL: https://issues.apache.org/jira/browse/KAFKA-15627

Re: [VOTE] KIP-951: Leader discovery optimisations for the client

2023-10-04 Thread Mayank Shekhar Narula
Summarising, there are 5 binding votes(Luke, Jose, Jun, David, Jason), and 1 non-binding vote(Kirk). With the current status of voting, KIP is accepted. Thanks again to all reviewers and voters. On Wed, Oct 4, 2023 at 9:37 AM Mayank Shekhar Narula < mayanks.nar...@gmail.com> wrote: &

Re: [VOTE] KIP-951: Leader discovery optimisations for the client

2023-10-04 Thread Mayank Shekhar Narula
> > > > Le mar. 3 oct. 2023 à 20:54, Jun Rao a écrit > : > > > > > Hi, Mayank, > > > > > > Thanks for the detailed explanation in the KIP. +1 from me. > > > > > > Jun > > > > > > On Wed, Sep 27, 2023 at 4:39 AM May

Re: Re: [DISCUSS] KIP-951: Leader discovery optimisations for the client

2023-10-02 Thread Mayank Shekhar Narula
s for all > current-leaders enumerated in PartitionData.". I suppose that this is > incorrect, right? We will only provide endpoints > when NOT_LEADER_OR_FOLLOWER or FENCED_LEADER_EPOCH are returned. The same > applies to the other schema. > > Best, > David > > > On Wed, Sep 27, 2

Re: Re: [DISCUSS] KIP-951: Leader discovery optimisations for the client

2023-09-27 Thread Mayank Shekhar Narula
the update Crispin - very helpful to have actual > > > performance > > > > > > data. 2-5% for the default configuration is a bit on the low side > > for > > > > > this > > > > > > kind of proposal. > > > > > > >

Re: [VOTE] KIP-951: Leader discovery optimisations for the client

2023-09-27 Thread Mayank Shekhar Narula
Reviving this thread, as the discussion thread has been updated. On Fri, Jul 28, 2023 at 11:29 AM Mayank Shekhar Narula < mayanks.nar...@gmail.com> wrote: > Thanks Jose. > > On Thu, Jul 27, 2023 at 5:46 PM José Armando García Sancio > wrote: > >> The KIP LGTM.

Re: [ANNOUNCE] New committer: Yash Mayya

2023-09-26 Thread Mayank Shekhar Narula
ask configurations > > > endpoint > > > KIP-980: Allow creating connectors in a stopped state > > > > > > Overall, Yash is known for insightful and friendly input to discussions > > > and his high quality contributions. > > > > > > Congratulations, Yash! > > > > > > Thanks, > > > > > > Bruno (on behalf of the Apache Kafka PMC) > -- Regards, Mayank Shekhar Narula

Re: [ANNOUNCE] New committer: Lucas Brutschy

2023-09-26 Thread Mayank Shekhar Narula
Tom Bentley wrote: > > > > > Congratulations! > > > > > > On Fri, 22 Sept 2023 at 09:11, Sophie Blee-Goldman < > ableegold...@gmail.com > > > > > > > wrote: > > > > > > > Congrats Lucas! > > > > > > > > -- Regards, Mayank Shekhar Narula

[jira] [Created] (KAFKA-15415) In Java-client, backoff should be skipped for retried producer-batch for a new leader

2023-08-29 Thread Mayank Shekhar Narula (Jira)
Mayank Shekhar Narula created KAFKA-15415: - Summary: In Java-client, backoff should be skipped for retried producer-batch for a new leader Key: KAFKA-15415 URL: https://issues.apache.org/jira/browse/KAFKA

Re: [VOTE] KIP-951: Leader discovery optimisations for the client

2023-07-28 Thread Mayank Shekhar Narula
Thanks Jose. On Thu, Jul 27, 2023 at 5:46 PM José Armando García Sancio wrote: > The KIP LGTM. Thanks for the design. I am looking forward to the > implementation. > > +1 (binding). > > Thanks! > -- > -José > -- Regards, Mayank Shekhar Narula

Re: Apache Kafka 3.6.0 release

2023-07-25 Thread Mayank Shekhar Narula
gt; > > > > > > > >> > > > > > > > > [1] - > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > >> > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-793%3A+Allow+sink+connectors+to+be+used+with+topic-mutating+SMTs > > >> > > > > > > > > > > >> > > > > > > > > Thanks, > > >> > > > > > > > > Yash > > >> > > > > > > > > > > >> > > > > > > > > On Mon, Jun 12, 2023 at 3:52 PM Satish Duggana < > > >> > > > > > > satish.dugg...@gmail.com > > >> > > > > > > > > > > >> > > > > > > > > wrote: > > >> > > > > > > > > > > >> > > > > > > > > > Hi, > > >> > > > > > > > > > I have created a release plan for Apache Kafka > version 3.6.0 on > > >> > > > > the > > >> > > > > > > > > > wiki. You can access the release plan and all > related > > >> > > > > information by > > >> > > > > > > > > > following this link: > > >> > > > > > > > > > > > >> > > > > > https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+3.6.0 > > >> > > > > > > > > > > > >> > > > > > > > > > The release plan outlines the key milestones and > important > > >> > > > dates > > >> > > > > for > > >> > > > > > > > > > version 3.6.0. Currently, the following dates have > been set for > > >> > > > > the > > >> > > > > > > > > > release: > > >> > > > > > > > > > > > >> > > > > > > > > > KIP Freeze: 26th July 23 > > >> > > > > > > > > > Feature Freeze : 16th Aug 23 > > >> > > > > > > > > > Code Freeze : 30th Aug 23 > > >> > > > > > > > > > > > >> > > > > > > > > > Please review the release plan and provide any > additional > > >> > > > > information > > >> > > > > > > > > > or updates regarding KIPs targeting version 3.6.0. > If you have > > >> > > > > > > > > > authored any KIPs that are missing a status or if > there are > > >> > > > > incorrect > > >> > > > > > > > > > status details, please make the necessary updates > and inform me > > >> > > > > so > > >> > > > > > > > > > that I can keep the plan accurate and up to date. > > >> > > > > > > > > > > > >> > > > > > > > > > Thanks, > > >> > > > > > > > > > Satish. > > >> > > > > > > > > > > > >> > > > > > > > > > On Mon, 17 Apr 2023 at 21:17, Luke Chen < > show...@gmail.com> > > >> > > > > wrote: > > >> > > > > > > > > > > > > >> > > > > > > > > > > Thanks for volunteering! > > >> > > > > > > > > > > > > >> > > > > > > > > > > +1 > > >> > > > > > > > > > > > > >> > > > > > > > > > > Luke > > >> > > > > > > > > > > > > >> > > > > > > > > > > On Mon, Apr 17, 2023 at 2:03 AM Ismael Juma < > > >> > > > ism...@juma.me.uk > > >> > > > > > > > >> > > > > > > > wrote: > > >> > > > > > > > > > > > > >> > > > > > > > > > > > Thanks for volunteering Satish. +1. > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > Ismael > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > On Sun, Apr 16, 2023 at 10:08 AM Satish Duggana > < > > >> > > > > > > > > > satish.dugg...@gmail.com> > > >> > > > > > > > > > > > wrote: > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > Hi, > > >> > > > > > > > > > > > > I would like to volunteer as release manager > for the next > > >> > > > > > > > release, > > >> > > > > > > > > > > > > which will be Apache Kafka 3.6.0. > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > If there are no objections, I will start a > release plan a > > >> > > > > week > > >> > > > > > > > after > > >> > > > > > > > > > > > > 3.5.0 release(around early May). > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > Thanks, > > >> > > > > > > > > > > > > Satish. > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > >> > > > > -- Regards, Mayank Shekhar Narula

[VOTE] KIP-951: Leader discovery optimisations for the client

2023-07-24 Thread Mayank Shekhar Narula
far! -- Regards, Mayank Shekhar Narula

Re: Re: [DISCUSS] KIP-951: Leader discovery optimisations for the client

2023-07-24 Thread Mayank Shekhar Narula
gt; Hi Mayank, > > We never backport new features to old releases. This new feature will be > only available from 3.6 (or 3.7) onwards for both client and server. > > Best, > David > > On Mon, Jul 24, 2023 at 5:20 PM Mayank Shekhar Narula < > mayanks.nar...@gmail.com>

Re: Re: [DISCUSS] KIP-951: Leader discovery optimisations for the client

2023-07-24 Thread Mayank Shekhar Narula
e > > clients and brokers? If not, can we set the version and taggedVersion > > to 15+? > > > > Thanks, > > -- > > -José > > > -- Regards, Mayank Shekhar Narula

Re: Re: [DISCUSS] KIP-951: Leader discovery optimisations for the client

2023-07-21 Thread Mayank Shekhar Narula
David 03. Fixed as well, to remove ignorable. In MetadataResponse, Rack came a version later, hence was marked ignorable. Thanks. On Fri, Jul 21, 2023 at 1:38 PM Mayank Shekhar Narula < mayanks.nar...@gmail.com> wrote: > Hi David > > 01. My reasoning noted in the KIP is tha

Re: Re: [DISCUSS] KIP-951: Leader discovery optimisations for the client

2023-07-21 Thread Mayank Shekhar Narula
s the field `Rack` have to really be ignorable? That does > not seem necessary to me but I may be wrong. > > Best, > David > > On Fri, Jul 21, 2023 at 12:32 AM Crispin Bernier > wrote: > > > Benchmark numbers have been posted on the KIP, please review. > > > >

Re: [DISCUSS] KIP-951: Leader discovery optimisations for the client

2023-07-20 Thread Mayank Shekhar Narula
s that it would be useful to see the > performance results. Otherwise, just a minor comment. If we don't plan to > bump up the FetchResponse version, we could just remove the reference to > version 16. > > Jun > > On Wed, Jul 19, 2023 at 2:31 PM Mayank Shekhar Narula < > m

Re: [DISCUSS] KIP-951: Leader discovery optimisations for the client

2023-07-19 Thread Mayank Shekhar Narula
ave the detail on top > > of my head, but I think we should lay out these behavioral changes. > > > > For #3: Thanks for the clarification. > > > > On Mon, Jul 17, 2023 at 10:39 AM Mayank Shekhar Narula < > > mayanks.nar...@gmail.com> wrote: > > > >

Re: [DISCUSS] KIP-951: Leader discovery optimisations for the client

2023-07-17 Thread Mayank Shekhar Narula
cess and I wonder if the current metadata response uses the > old leader or null as the leader isn't readily available yet. > > Thanks, > P > > On Fri, Jul 14, 2023 at 11:30 AM Kirk True wrote: > > > Hi Mayank, > > > > > On Jul 14,

Re: [DISCUSS] KIP-951: Leader discovery optimisations for the client

2023-07-17 Thread Mayank Shekhar Narula
. Would the fourth attempt > (with the new leader) still be performed without any delay? To be honest, > I’m not sure that case is valid, but I would assume it would retry > immediately, right? > > Thanks, > Kirk > > > On Jul 13, 2023, at 7:15 AM, Mayank Shekhar Narula &l

Re: Re: [DISCUSS] KIP-951: Leader discovery optimisations for the client

2023-07-17 Thread Mayank Shekhar Narula
Hi Mayank, > > If we bump the version, the broker can tell whether it’s worth > providing the leader > > endpoint information to the client when the leader has changed. > That’s my reasoning. > > > > Thanks, > > Andrew > > > > > On 13 Jul 2

Re: [DISCUSS] KIP-951: Leader discovery optimisations for the client

2023-07-14 Thread Mayank Shekhar Narula
change you’re referring to? The idea of > non-brokers serving as replicas is blowing my mind a bit :) > > Jose, I missed this as well, would love to know more about non-broker serving as replica! -- Regards, Mayank Shekhar Narula

Re: [DISCUSS] KIP-951: Leader discovery optimisations for the client

2023-07-14 Thread Mayank Shekhar Narula
his array is a mapping of replica/node > id to their endpoint. For example, in the future we may use it to send > endpoint information when sending the replica id to fetch from a > follower. > > Thanks, > -- > -José > -- Regards, Mayank Shekhar Narula

Re: [DISCUSS] KIP-951: Leader discovery optimisations for the client

2023-07-14 Thread Mayank Shekhar Narula
Hi Ismael Right now, effort is to get baseline Vs KIP-proposed changes #s. After that will look at the alternative #s. On Fri, Jul 14, 2023 at 12:38 AM Ismael Juma wrote: > Hi Mayank, > > See my answer below. > > On Thu, Jul 13, 2023 at 10:24 AM Mayank Shekhar Narula

Re: [DISCUSS] KIP-951: Leader discovery optimisations for the client

2023-07-13 Thread Mayank Shekhar Narula
rnative too. > 3, It's wasteful to repeat the leader node information for every partition > - we should probably have a separate list or map with the node information. > Looks like Jose suggested the same/similar. > > Ismael > > On Thu, Jul 13, 2023 at 7:16 AM Mayank Shekhar

Re: [DISCUSS] KIP-951: Leader discovery optimisations for the client

2023-07-13 Thread Mayank Shekhar Narula
ut, I > think it’s important to > >> know which partitions are now lead by which node. That information at > least needs to be > >> per-partition I think. I may have misunderstood, but it sounded like > your comment > >> suggestion lost that relationship. > > > > Each partition in both the FETCH response and the PRODUCE response > > will have the CurrentLeader, the tuple leader id and leader epoch. > > Clients can use this information to update their partition to leader > > id and leader epoch mapping. > > > > They can also use the NodeEndpoints to update their mapping from > > replica id to the tuple host, port and rack so that they can connect > > to the correct node for future FETCH requests and PRODUCE requests. > > > > Thanks, > > -- > > -José > > -- Regards, Mayank Shekhar Narula

[DISCUSS] KIP-951: Leader discovery optimisations for the client

2023-07-13 Thread Mayank Shekhar Narula
Hi everyone Following KIP is up for discussion. Thanks for your feedback. https://cwiki.apache.org/confluence/display/KAFKA/KIP-951%3A+Leader+discovery+optimisations+for+the+client -- Regards, Mayank Shekhar Narula

Re: Requesting permissions to contribute to Apache Kafka

2023-06-30 Thread Mayank Shekhar Narula
thanks Divij! On Thu, Jun 29, 2023 at 8:19 PM Divij Vaidya wrote: > You should be all set. > > -- > Divij Vaidya > > > > On Thu, Jun 29, 2023 at 8:45 PM Mayank Shekhar Narula < > mayanks.nar...@gmail.com> wrote: > > > - can someone gran

Requesting permissions to contribute to Apache Kafka

2023-06-29 Thread Mayank Shekhar Narula
as Wiki id does have ".". -- Regards, Mayank Shekhar Narula