Re: [VOTE] KIP-951: Leader discovery optimisations for the client
David Thanks, i have shared it here https://lists.apache.org/thread/0fd0og8j6gj9b2wtppnpy7fz8pzzv20r On Tue, Feb 6, 2024 at 4:13 PM David Jacot wrote: > Hi, > > Thanks for bringing this up. It may be worth bringing it in the 3.7 release > thread too as it may qualify as a blocker for the release. > > Best, > David > > On Tue, Feb 6, 2024 at 5:09 PM Mayank Shekhar Narula < > mayanks.nar...@gmail.com> wrote: > > > Hi Folks > > > > KIP-951 was delivered fully in AK 3.7. Its 1st optimisation was delivered > > in 3.6.1, to skip backoff period for a produce batch being retried to new > > leader i.e. KAFKA-15415. > > > > KAFKA-15415 current implementation introduced a performance regression, > by > > increasing synchronization on the produce path, especially for high > > partition counts. The description section of > > https://issues.apache.org/jira/browse/KAFKA-16226 goes more into details > > of > > the regression. > > > > I have put up a fix https://github.com/apache/kafka/pull/15323, which > > removes this synchronization. The fix adds a new public method to > > Cluster.java, and a public constructor to PartitionInfo.java. Let me know > > your thoughts. > > > > On Wed, Oct 4, 2023 at 10:09 AM Mayank Shekhar Narula < > > mayanks.nar...@gmail.com> wrote: > > > > > Summarising, there are 5 binding votes(Luke, Jose, Jun, David, Jason), > > and > > > 1 non-binding vote(Kirk). > > > > > > With the current status of voting, KIP is accepted. > > > > > > Thanks again to all reviewers and voters. > > > > > > > > > > > > On Wed, Oct 4, 2023 at 9:37 AM Mayank Shekhar Narula < > > > mayanks.nar...@gmail.com> wrote: > > > > > >> Thank you all for your votes, Jun, David, and Jason! > > >> > > >> On Tue, Oct 3, 2023 at 11:44 PM Jason Gustafson > > >> wrote: > > >> > > >>> +1 Thanks for the KIP > > >>> > > >>> On Tue, Oct 3, 2023 at 12:30 PM David Jacot > > >>> wrote: > > >>> > > >>> > Thanks for the KIP. +1 from me as well. > > >>> > > > >>> > Best, > > >>> > David > > >>> > > > >>> > Le mar. 3 oct. 2023 à 20:54, Jun Rao a > > >>> écrit : > > >>> > > > >>> > > Hi, Mayank, > > >>> > > > > >>> > > Thanks for the detailed explanation in the KIP. +1 from me. > > >>> > > > > >>> > > Jun > > >>> > > > > >>> > > On Wed, Sep 27, 2023 at 4:39 AM Mayank Shekhar Narula < > > >>> > > mayanks.nar...@gmail.com> wrote: > > >>> > > > > >>> > > > Reviving this thread, as the discussion thread has been > updated. > > >>> > > > > > >>> > > > On Fri, Jul 28, 2023 at 11:29 AM Mayank Shekhar Narula < > > >>> > > > mayanks.nar...@gmail.com> wrote: > > >>> > > > > > >>> > > > > Thanks Jose. > > >>> > > > > > > >>> > > > > On Thu, Jul 27, 2023 at 5:46 PM José Armando García Sancio > > >>> > > > > wrote: > > >>> > > > > > > >>> > > > >> The KIP LGTM. Thanks for the design. I am looking forward to > > the > > >>> > > > >> implementation. > > >>> > > > >> > > >>> > > > >> +1 (binding). > > >>> > > > >> > > >>> > > > >> Thanks! > > >>> > > > >> -- > > >>> > > > >> -José > > >>> > > > >> > > >>> > > > > > > >>> > > > > > > >>> > > > > -- > > >>> > > > > Regards, > > >>> > > > > Mayank Shekhar Narula > > >>> > > > > > > >>> > > > > > >>> > > > > > >>> > > > -- > > >>> > > > Regards, > > >>> > > > Mayank Shekhar Narula > > >>> > > > > > >>> > > > > >>> > > > >>> > > >> > > >> > > >> -- > > >> Regards, > > >> Mayank Shekhar Narula > > >> > > > > > > > > > -- > > > Regards, > > > Mayank Shekhar Narula > > > > > > > > > -- > > Regards, > > Mayank Shekhar Narula > > > -- Regards, Mayank Shekhar Narula
Re: [VOTE] KIP-951: Leader discovery optimisations for the client
Hi, Thanks for bringing this up. It may be worth bringing it in the 3.7 release thread too as it may qualify as a blocker for the release. Best, David On Tue, Feb 6, 2024 at 5:09 PM Mayank Shekhar Narula < mayanks.nar...@gmail.com> wrote: > Hi Folks > > KIP-951 was delivered fully in AK 3.7. Its 1st optimisation was delivered > in 3.6.1, to skip backoff period for a produce batch being retried to new > leader i.e. KAFKA-15415. > > KAFKA-15415 current implementation introduced a performance regression, by > increasing synchronization on the produce path, especially for high > partition counts. The description section of > https://issues.apache.org/jira/browse/KAFKA-16226 goes more into details > of > the regression. > > I have put up a fix https://github.com/apache/kafka/pull/15323, which > removes this synchronization. The fix adds a new public method to > Cluster.java, and a public constructor to PartitionInfo.java. Let me know > your thoughts. > > On Wed, Oct 4, 2023 at 10:09 AM Mayank Shekhar Narula < > mayanks.nar...@gmail.com> wrote: > > > Summarising, there are 5 binding votes(Luke, Jose, Jun, David, Jason), > and > > 1 non-binding vote(Kirk). > > > > With the current status of voting, KIP is accepted. > > > > Thanks again to all reviewers and voters. > > > > > > > > On Wed, Oct 4, 2023 at 9:37 AM Mayank Shekhar Narula < > > mayanks.nar...@gmail.com> wrote: > > > >> Thank you all for your votes, Jun, David, and Jason! > >> > >> On Tue, Oct 3, 2023 at 11:44 PM Jason Gustafson > >> wrote: > >> > >>> +1 Thanks for the KIP > >>> > >>> On Tue, Oct 3, 2023 at 12:30 PM David Jacot > >>> wrote: > >>> > >>> > Thanks for the KIP. +1 from me as well. > >>> > > >>> > Best, > >>> > David > >>> > > >>> > Le mar. 3 oct. 2023 à 20:54, Jun Rao a > >>> écrit : > >>> > > >>> > > Hi, Mayank, > >>> > > > >>> > > Thanks for the detailed explanation in the KIP. +1 from me. > >>> > > > >>> > > Jun > >>> > > > >>> > > On Wed, Sep 27, 2023 at 4:39 AM Mayank Shekhar Narula < > >>> > > mayanks.nar...@gmail.com> wrote: > >>> > > > >>> > > > Reviving this thread, as the discussion thread has been updated. > >>> > > > > >>> > > > On Fri, Jul 28, 2023 at 11:29 AM Mayank Shekhar Narula < > >>> > > > mayanks.nar...@gmail.com> wrote: > >>> > > > > >>> > > > > Thanks Jose. > >>> > > > > > >>> > > > > On Thu, Jul 27, 2023 at 5:46 PM José Armando García Sancio > >>> > > > > wrote: > >>> > > > > > >>> > > > >> The KIP LGTM. Thanks for the design. I am looking forward to > the > >>> > > > >> implementation. > >>> > > > >> > >>> > > > >> +1 (binding). > >>> > > > >> > >>> > > > >> Thanks! > >>> > > > >> -- > >>> > > > >> -José > >>> > > > >> > >>> > > > > > >>> > > > > > >>> > > > > -- > >>> > > > > Regards, > >>> > > > > Mayank Shekhar Narula > >>> > > > > > >>> > > > > >>> > > > > >>> > > > -- > >>> > > > Regards, > >>> > > > Mayank Shekhar Narula > >>> > > > > >>> > > > >>> > > >>> > >> > >> > >> -- > >> Regards, > >> Mayank Shekhar Narula > >> > > > > > > -- > > Regards, > > Mayank Shekhar Narula > > > > > -- > Regards, > Mayank Shekhar Narula >
Re: [VOTE] KIP-951: Leader discovery optimisations for the client
Hi Folks KIP-951 was delivered fully in AK 3.7. Its 1st optimisation was delivered in 3.6.1, to skip backoff period for a produce batch being retried to new leader i.e. KAFKA-15415. KAFKA-15415 current implementation introduced a performance regression, by increasing synchronization on the produce path, especially for high partition counts. The description section of https://issues.apache.org/jira/browse/KAFKA-16226 goes more into details of the regression. I have put up a fix https://github.com/apache/kafka/pull/15323, which removes this synchronization. The fix adds a new public method to Cluster.java, and a public constructor to PartitionInfo.java. Let me know your thoughts. On Wed, Oct 4, 2023 at 10:09 AM Mayank Shekhar Narula < mayanks.nar...@gmail.com> wrote: > Summarising, there are 5 binding votes(Luke, Jose, Jun, David, Jason), and > 1 non-binding vote(Kirk). > > With the current status of voting, KIP is accepted. > > Thanks again to all reviewers and voters. > > > > On Wed, Oct 4, 2023 at 9:37 AM Mayank Shekhar Narula < > mayanks.nar...@gmail.com> wrote: > >> Thank you all for your votes, Jun, David, and Jason! >> >> On Tue, Oct 3, 2023 at 11:44 PM Jason Gustafson >> wrote: >> >>> +1 Thanks for the KIP >>> >>> On Tue, Oct 3, 2023 at 12:30 PM David Jacot >>> wrote: >>> >>> > Thanks for the KIP. +1 from me as well. >>> > >>> > Best, >>> > David >>> > >>> > Le mar. 3 oct. 2023 à 20:54, Jun Rao a >>> écrit : >>> > >>> > > Hi, Mayank, >>> > > >>> > > Thanks for the detailed explanation in the KIP. +1 from me. >>> > > >>> > > Jun >>> > > >>> > > On Wed, Sep 27, 2023 at 4:39 AM Mayank Shekhar Narula < >>> > > mayanks.nar...@gmail.com> wrote: >>> > > >>> > > > Reviving this thread, as the discussion thread has been updated. >>> > > > >>> > > > On Fri, Jul 28, 2023 at 11:29 AM Mayank Shekhar Narula < >>> > > > mayanks.nar...@gmail.com> wrote: >>> > > > >>> > > > > Thanks Jose. >>> > > > > >>> > > > > On Thu, Jul 27, 2023 at 5:46 PM José Armando García Sancio >>> > > > > wrote: >>> > > > > >>> > > > >> The KIP LGTM. Thanks for the design. I am looking forward to the >>> > > > >> implementation. >>> > > > >> >>> > > > >> +1 (binding). >>> > > > >> >>> > > > >> Thanks! >>> > > > >> -- >>> > > > >> -José >>> > > > >> >>> > > > > >>> > > > > >>> > > > > -- >>> > > > > Regards, >>> > > > > Mayank Shekhar Narula >>> > > > > >>> > > > >>> > > > >>> > > > -- >>> > > > Regards, >>> > > > Mayank Shekhar Narula >>> > > > >>> > > >>> > >>> >> >> >> -- >> Regards, >> Mayank Shekhar Narula >> > > > -- > Regards, > Mayank Shekhar Narula > -- Regards, Mayank Shekhar Narula
Re: [VOTE] KIP-951: Leader discovery optimisations for the client
Summarising, there are 5 binding votes(Luke, Jose, Jun, David, Jason), and 1 non-binding vote(Kirk). With the current status of voting, KIP is accepted. Thanks again to all reviewers and voters. On Wed, Oct 4, 2023 at 9:37 AM Mayank Shekhar Narula < mayanks.nar...@gmail.com> wrote: > Thank you all for your votes, Jun, David, and Jason! > > On Tue, Oct 3, 2023 at 11:44 PM Jason Gustafson > wrote: > >> +1 Thanks for the KIP >> >> On Tue, Oct 3, 2023 at 12:30 PM David Jacot >> wrote: >> >> > Thanks for the KIP. +1 from me as well. >> > >> > Best, >> > David >> > >> > Le mar. 3 oct. 2023 à 20:54, Jun Rao a >> écrit : >> > >> > > Hi, Mayank, >> > > >> > > Thanks for the detailed explanation in the KIP. +1 from me. >> > > >> > > Jun >> > > >> > > On Wed, Sep 27, 2023 at 4:39 AM Mayank Shekhar Narula < >> > > mayanks.nar...@gmail.com> wrote: >> > > >> > > > Reviving this thread, as the discussion thread has been updated. >> > > > >> > > > On Fri, Jul 28, 2023 at 11:29 AM Mayank Shekhar Narula < >> > > > mayanks.nar...@gmail.com> wrote: >> > > > >> > > > > Thanks Jose. >> > > > > >> > > > > On Thu, Jul 27, 2023 at 5:46 PM José Armando García Sancio >> > > > > wrote: >> > > > > >> > > > >> The KIP LGTM. Thanks for the design. I am looking forward to the >> > > > >> implementation. >> > > > >> >> > > > >> +1 (binding). >> > > > >> >> > > > >> Thanks! >> > > > >> -- >> > > > >> -José >> > > > >> >> > > > > >> > > > > >> > > > > -- >> > > > > Regards, >> > > > > Mayank Shekhar Narula >> > > > > >> > > > >> > > > >> > > > -- >> > > > Regards, >> > > > Mayank Shekhar Narula >> > > > >> > > >> > >> > > > -- > Regards, > Mayank Shekhar Narula > -- Regards, Mayank Shekhar Narula
Re: [VOTE] KIP-951: Leader discovery optimisations for the client
Thank you all for your votes, Jun, David, and Jason! On Tue, Oct 3, 2023 at 11:44 PM Jason Gustafson wrote: > +1 Thanks for the KIP > > On Tue, Oct 3, 2023 at 12:30 PM David Jacot wrote: > > > Thanks for the KIP. +1 from me as well. > > > > Best, > > David > > > > Le mar. 3 oct. 2023 à 20:54, Jun Rao a écrit > : > > > > > Hi, Mayank, > > > > > > Thanks for the detailed explanation in the KIP. +1 from me. > > > > > > Jun > > > > > > On Wed, Sep 27, 2023 at 4:39 AM Mayank Shekhar Narula < > > > mayanks.nar...@gmail.com> wrote: > > > > > > > Reviving this thread, as the discussion thread has been updated. > > > > > > > > On Fri, Jul 28, 2023 at 11:29 AM Mayank Shekhar Narula < > > > > mayanks.nar...@gmail.com> wrote: > > > > > > > > > Thanks Jose. > > > > > > > > > > On Thu, Jul 27, 2023 at 5:46 PM José Armando García Sancio > > > > > wrote: > > > > > > > > > >> The KIP LGTM. Thanks for the design. I am looking forward to the > > > > >> implementation. > > > > >> > > > > >> +1 (binding). > > > > >> > > > > >> Thanks! > > > > >> -- > > > > >> -José > > > > >> > > > > > > > > > > > > > > > -- > > > > > Regards, > > > > > Mayank Shekhar Narula > > > > > > > > > > > > > > > > > -- > > > > Regards, > > > > Mayank Shekhar Narula > > > > > > > > > > -- Regards, Mayank Shekhar Narula
Re: [VOTE] KIP-951: Leader discovery optimisations for the client
+1 Thanks for the KIP On Tue, Oct 3, 2023 at 12:30 PM David Jacot wrote: > Thanks for the KIP. +1 from me as well. > > Best, > David > > Le mar. 3 oct. 2023 à 20:54, Jun Rao a écrit : > > > Hi, Mayank, > > > > Thanks for the detailed explanation in the KIP. +1 from me. > > > > Jun > > > > On Wed, Sep 27, 2023 at 4:39 AM Mayank Shekhar Narula < > > mayanks.nar...@gmail.com> wrote: > > > > > Reviving this thread, as the discussion thread has been updated. > > > > > > On Fri, Jul 28, 2023 at 11:29 AM Mayank Shekhar Narula < > > > mayanks.nar...@gmail.com> wrote: > > > > > > > Thanks Jose. > > > > > > > > On Thu, Jul 27, 2023 at 5:46 PM José Armando García Sancio > > > > wrote: > > > > > > > >> The KIP LGTM. Thanks for the design. I am looking forward to the > > > >> implementation. > > > >> > > > >> +1 (binding). > > > >> > > > >> Thanks! > > > >> -- > > > >> -José > > > >> > > > > > > > > > > > > -- > > > > Regards, > > > > Mayank Shekhar Narula > > > > > > > > > > > > > -- > > > Regards, > > > Mayank Shekhar Narula > > > > > >
Re: [VOTE] KIP-951: Leader discovery optimisations for the client
Thanks for the KIP. +1 from me as well. Best, David Le mar. 3 oct. 2023 à 20:54, Jun Rao a écrit : > Hi, Mayank, > > Thanks for the detailed explanation in the KIP. +1 from me. > > Jun > > On Wed, Sep 27, 2023 at 4:39 AM Mayank Shekhar Narula < > mayanks.nar...@gmail.com> wrote: > > > Reviving this thread, as the discussion thread has been updated. > > > > On Fri, Jul 28, 2023 at 11:29 AM Mayank Shekhar Narula < > > mayanks.nar...@gmail.com> wrote: > > > > > Thanks Jose. > > > > > > On Thu, Jul 27, 2023 at 5:46 PM José Armando García Sancio > > > wrote: > > > > > >> The KIP LGTM. Thanks for the design. I am looking forward to the > > >> implementation. > > >> > > >> +1 (binding). > > >> > > >> Thanks! > > >> -- > > >> -José > > >> > > > > > > > > > -- > > > Regards, > > > Mayank Shekhar Narula > > > > > > > > > -- > > Regards, > > Mayank Shekhar Narula > > >
Re: [VOTE] KIP-951: Leader discovery optimisations for the client
Hi, Mayank, Thanks for the detailed explanation in the KIP. +1 from me. Jun On Wed, Sep 27, 2023 at 4:39 AM Mayank Shekhar Narula < mayanks.nar...@gmail.com> wrote: > Reviving this thread, as the discussion thread has been updated. > > On Fri, Jul 28, 2023 at 11:29 AM Mayank Shekhar Narula < > mayanks.nar...@gmail.com> wrote: > > > Thanks Jose. > > > > On Thu, Jul 27, 2023 at 5:46 PM José Armando García Sancio > > wrote: > > > >> The KIP LGTM. Thanks for the design. I am looking forward to the > >> implementation. > >> > >> +1 (binding). > >> > >> Thanks! > >> -- > >> -José > >> > > > > > > -- > > Regards, > > Mayank Shekhar Narula > > > > > -- > Regards, > Mayank Shekhar Narula >
Re: [VOTE] KIP-951: Leader discovery optimisations for the client
Reviving this thread, as the discussion thread has been updated. On Fri, Jul 28, 2023 at 11:29 AM Mayank Shekhar Narula < mayanks.nar...@gmail.com> wrote: > Thanks Jose. > > On Thu, Jul 27, 2023 at 5:46 PM José Armando García Sancio > wrote: > >> The KIP LGTM. Thanks for the design. I am looking forward to the >> implementation. >> >> +1 (binding). >> >> Thanks! >> -- >> -José >> > > > -- > Regards, > Mayank Shekhar Narula > -- Regards, Mayank Shekhar Narula
Re: [VOTE] KIP-951: Leader discovery optimisations for the client
Thanks Jose. On Thu, Jul 27, 2023 at 5:46 PM José Armando García Sancio wrote: > The KIP LGTM. Thanks for the design. I am looking forward to the > implementation. > > +1 (binding). > > Thanks! > -- > -José > -- Regards, Mayank Shekhar Narula
Re: [VOTE] KIP-951: Leader discovery optimisations for the client
The KIP LGTM. Thanks for the design. I am looking forward to the implementation. +1 (binding). Thanks! -- -José
Re: [VOTE] KIP-951: Leader discovery optimisations for the client
+1 (non-binding) Thanks Mayank! On Mon, Jul 24, 2023, at 11:24 AM, Mayank Shekhar Narula wrote: > Hi all > > Opening the voting process for this KIP, given the deadline for KIP > acceptance for 3.6 is impending. Open to keep the discussion going if the > community needs more time to converge. > > KIP -> https://cwiki.apache.org/confluence/x/F4uzDw > > Thanks for participating in the discussion so far! > > -- > Regards, > Mayank Shekhar Narula >
[VOTE] KIP-951: Leader discovery optimisations for the client
Hi all Opening the voting process for this KIP, given the deadline for KIP acceptance for 3.6 is impending. Open to keep the discussion going if the community needs more time to converge. KIP -> https://cwiki.apache.org/confluence/x/F4uzDw Thanks for participating in the discussion so far! -- Regards, Mayank Shekhar Narula
