Re: [kafka-clients] [VOTE] 3.6.0 RC1

[Luke Chen]

Hi Satish,

I verified with:
1. Ran quick start in KRaft for scala 2.12 artifact
2. Making sure the checksum are correct
3. Browsing release notes, documents, javadocs, protocols.

I filed KAFKA-15491 for
log output improvement while testing stream application.
It won't be blocker in v3.6.0.

For KAFKA-15489 , I'm
fine if we decide to fix it in v3.6.1/v3.7.0.

+1 (binding) from me.

Thank you.
Luke

On Sun, Sep 24, 2023 at 3:38 AM Ismael Juma  wrote:

> Given that this is not a regression and there have been no reports for over
> a year, I think it's ok for this to land in 3.6.1.
>
> Ismael
>
> On Sat, Sep 23, 2023 at 9:32 AM Satish Duggana 
> wrote:
>
> > Thanks Luke for reporting KRaft issue[1].
> >
> > I am not sure whether it is a release blocker for 3.6.0. Need input
> > from other KRaft experts also to finalize the decision. Even if we
> > adopt a fix, do not we need to bake it for some time before it is
> > pushed to production to avoid any regressions as this change is in the
> > critical paths?
> >
> > 1. https://issues.apache.org/jira/browse/KAFKA-15489
> >
> > Thanks,
> > Satish.
> >
> > On Sat, 23 Sept 2023 at 03:08, Luke Chen  wrote:
> > >
> > > Hi Satish,
> > >
> > > I found the current KRaft implementation will have "split brain" issue
> > when
> > > network partition happens, which will cause inconsistent metadata
> > returned
> > > from the controller.
> > > Filed KAFKA-15489 
> > for
> > > this issue, and PR  is
> ready
> > > for review.
> > >
> > > Even though this is not a regression issue (this has already existed
> > since
> > > the 1st release of KRaft feature), I think this is an important issue
> > since
> > > KRaft is announced production ready.
> > > Not sure what other people's thoughts are.
> > >
> > > Thank you.
> > > Luke
> > >
> > > On Thu, Sep 21, 2023 at 6:33 PM Josep Prat  >
> > > wrote:
> > >
> > > > Hi Satish,
> > > >
> > > > I ran the following validation steps:
> > > > - Built from source with Java 11 and Scala 2.13
> > > > - Verified Signatures and hashes of the artifacts generated
> > > > - Navigated through Javadoc including links to JDK classes
> > > > - Run the unit tests
> > > > - Run integration tests
> > > > - Run the quickstart in KRaft and Zookeeper mode
> > > >
> > > >
> > > > I +1 this release (non-binding)
> > > >
> > > > Thanks for your efforts!
> > > >
> > > > On Thu, Sep 21, 2023 at 2:59 AM Satish Duggana <
> > satish.dugg...@gmail.com>
> > > > wrote:
> > > >
> > > > > Thanks Greg for verifying the release including the earlier
> > > > > blocker(KAFKA-15473) verification.
> > > > >
> > > > > ~Satish.
> > > > >
> > > > > On Wed, 20 Sept 2023 at 22:30, Greg Harris
> >  > > > >
> > > > > wrote:
> > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > I verified the functionality of KIP-898 and the recent fix for
> > > > > > KAFKA-15473 with the following steps:
> > > > > >
> > > > > > 1. I started a 3.5.1 broker, and a 3.5.1 worker with most (>400)
> > > > > > publicly available plugins installed
> > > > > > 2. I captured the output of /connector-plugins
> > > > > > 3. I upgraded the worker to 3.6.0-rc1
> > > > > > 4. I captured the output of /connector-plugins with various
> > settings
> > > > > > of plugin.discovery
> > > > > > 5. I ran the migration script to add manifests to my plugins
> > > > > > 6. I captured the output of /connector-plugins with various
> > settings
> > > > > > of plugin.discovery
> > > > > > 7. I downgraded the worker to 3.5.1
> > > > > > 8. I diffed the output of /connector-plugins across the different
> > > > > > cases and observed the expected changes.
> > > > > > a. When plugins are migrated for 3.6.0, all modes produce
> > identical
> > > > > > results.
> > > > > > b. When plugins are not migrated for 3.6.0, only_scan and
> > > > > > hybrid_warn produce identical results, hybrid_fail crashes, and
> > > > > > service_load is missing plugins
> > > > > > c. When upgrading from 3.5.1 I see that plugins with invalid
> > > > > > constructors are hidden, AK plugins now have versions,
> > multi-interface
> > > > > > plugins now show each interface type, and plugins using
> > AppInfoParser
> > > > > > change versions.
> > > > > > d. The startup logs now include descriptive errors for
> invalid
> > > > > > plugins that otherwise would have been thrown at runtime
> > > > > > d. The fix for KAFKA-15473 prevents duplicates
> > > > > > e. The output for 3.5.1 after downgrading is identical to
> > before.
> > > > > >
> > > > > > +1 (non-binding)
> > > > > >
> > > > > > Thanks Satish for running the release!
> > > > > >
> > > > > > On Wed, Sep 20, 2023 at 8:36 AM Divij Vaidya 
> > > > wrote:
> > > > > > >
> > > > > > > Hey Satish
> > > > > > >
> > > > > > > My comments about documentation misses from RC0 vote thread [1]
> > are

[jira] [Created] (KAFKA-15491) RackId doesn't exist while running WordCountDemo

[Luke Chen (Jira)]

Luke Chen created KAFKA-15491:
-

 Summary: RackId doesn't exist while running WordCountDemo
 Key: KAFKA-15491
 URL: https://issues.apache.org/jira/browse/KAFKA-15491
 Project: Kafka
  Issue Type: Bug
  Components: streams
Reporter: Luke Chen


While running the WordCountDemo following the 
[docs|https://kafka.apache.org/documentation/streams/quickstart], I saw the 
following error logs in the stream application output. Though everything still 
works fine, it'd be better there are no ERROR logs in the demo app.


{code:java}
[2023-09-24 14:15:11,723] ERROR RackId doesn't exist for process 
e2391098-23e8-47eb-8d5e-ff6e697c33f5 and consumer 
streams-wordcount-e2391098-23e8-47eb-8d5e-ff6e697c33f5-StreamThread-1-consumer-adae58be-f5f5-429b-a2b4-67bf732726e8
 (org.apache.kafka.streams.processor.internals.assignment.RackAwareTaskAssignor)
[2023-09-24 14:15:11,757] ERROR RackId doesn't exist for process 
e2391098-23e8-47eb-8d5e-ff6e697c33f5 and consumer 
streams-wordcount-e2391098-23e8-47eb-8d5e-ff6e697c33f5-StreamThread-1-consumer-adae58be-f5f5-429b-a2b4-67bf732726e8
 (org.apache.kafka.streams.processor.internals.assignment.RackAwareTaskAssignor)
{code}




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [ANNOUNCE] New Kafka PMC Member: Justine Olshan

[Sophie Blee-Goldman]

Congrats Justine!

On Sat, Sep 23, 2023, 4:36 PM Tom Bentley  wrote:

> Congratulations!
>
> On Sun, 24 Sept 2023 at 12:32, Satish Duggana 
> wrote:
>
> > Congratulations Justine!!
> >
> > On Sat, 23 Sept 2023 at 15:46, Bill Bejeck  wrote:
> > >
> > > Congrats Justine!
> > >
> > > -Bill
> > >
> > > On Sat, Sep 23, 2023 at 6:23 PM Greg Harris
>  > >
> > > wrote:
> > >
> > > > Congratulations Justine!
> > > >
> > > > On Sat, Sep 23, 2023 at 5:49 AM Boudjelda Mohamed Said
> > > >  wrote:
> > > > >
> > > > > Congrats Justin !
> > > > >
> > > > > On Sat 23 Sep 2023 at 14:44, Randall Hauch 
> wrote:
> > > > >
> > > > > > Congratulations, Justine!
> > > > > >
> > > > > > On Sat, Sep 23, 2023 at 4:25 AM Kamal Chandraprakash <
> > > > > > kamal.chandraprak...@gmail.com> wrote:
> > > > > >
> > > > > > > Congrats Justine!
> > > > > > >
> > > > > > > On Sat, Sep 23, 2023, 13:28 Divij Vaidya <
> > divijvaidy...@gmail.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > > Congratulations Justine!
> > > > > > > >
> > > > > > > > On Sat 23. Sep 2023 at 07:06, Chris Egerton <
> > > > fearthecel...@gmail.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Congrats Justine!
> > > > > > > > > On Fri, Sep 22, 2023, 20:47 Guozhang Wang <
> > > > > > guozhang.wang...@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Congratulations!
> > > > > > > > > >
> > > > > > > > > > On Fri, Sep 22, 2023 at 8:44 PM Tzu-Li (Gordon) Tai <
> > > > > > > > tzuli...@apache.org
> > > > > > > > > >
> > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > Congratulations Justine!
> > > > > > > > > > >
> > > > > > > > > > > On Fri, Sep 22, 2023, 19:25 Philip Nee <
> > philip...@gmail.com>
> > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Congrats Justine!
> > > > > > > > > > > >
> > > > > > > > > > > > On Fri, Sep 22, 2023 at 7:07 PM Luke Chen <
> > > > show...@gmail.com>
> > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > Hi, Everyone,
> > > > > > > > > > > > >
> > > > > > > > > > > > > Justine Olshan has been a Kafka committer since
> Dec.
> > > > 2022.
> > > > > > She
> > > > > > > > has
> > > > > > > > > > been
> > > > > > > > > > > > > very active and instrumental to the community since
> > > > becoming
> > > > > > a
> > > > > > > > > > committer.
> > > > > > > > > > > > > It's my pleasure to announce that Justine is now a
> > > > member of
> > > > > > > > Kafka
> > > > > > > > > > PMC.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Congratulations Justine!
> > > > > > > > > > > > >
> > > > > > > > > > > > > Luke
> > > > > > > > > > > > > on behalf of Apache Kafka PMC
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > >
> >
> >
>

Re: [ANNOUNCE] New Kafka PMC Member: Justine Olshan

[Tom Bentley]

Congratulations!

On Sun, 24 Sept 2023 at 12:32, Satish Duggana 
wrote:

> Congratulations Justine!!
>
> On Sat, 23 Sept 2023 at 15:46, Bill Bejeck  wrote:
> >
> > Congrats Justine!
> >
> > -Bill
> >
> > On Sat, Sep 23, 2023 at 6:23 PM Greg Harris  >
> > wrote:
> >
> > > Congratulations Justine!
> > >
> > > On Sat, Sep 23, 2023 at 5:49 AM Boudjelda Mohamed Said
> > >  wrote:
> > > >
> > > > Congrats Justin !
> > > >
> > > > On Sat 23 Sep 2023 at 14:44, Randall Hauch  wrote:
> > > >
> > > > > Congratulations, Justine!
> > > > >
> > > > > On Sat, Sep 23, 2023 at 4:25 AM Kamal Chandraprakash <
> > > > > kamal.chandraprak...@gmail.com> wrote:
> > > > >
> > > > > > Congrats Justine!
> > > > > >
> > > > > > On Sat, Sep 23, 2023, 13:28 Divij Vaidya <
> divijvaidy...@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > > Congratulations Justine!
> > > > > > >
> > > > > > > On Sat 23. Sep 2023 at 07:06, Chris Egerton <
> > > fearthecel...@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Congrats Justine!
> > > > > > > > On Fri, Sep 22, 2023, 20:47 Guozhang Wang <
> > > > > guozhang.wang...@gmail.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Congratulations!
> > > > > > > > >
> > > > > > > > > On Fri, Sep 22, 2023 at 8:44 PM Tzu-Li (Gordon) Tai <
> > > > > > > tzuli...@apache.org
> > > > > > > > >
> > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > Congratulations Justine!
> > > > > > > > > >
> > > > > > > > > > On Fri, Sep 22, 2023, 19:25 Philip Nee <
> philip...@gmail.com>
> > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Congrats Justine!
> > > > > > > > > > >
> > > > > > > > > > > On Fri, Sep 22, 2023 at 7:07 PM Luke Chen <
> > > show...@gmail.com>
> > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Hi, Everyone,
> > > > > > > > > > > >
> > > > > > > > > > > > Justine Olshan has been a Kafka committer since Dec.
> > > 2022.
> > > > > She
> > > > > > > has
> > > > > > > > > been
> > > > > > > > > > > > very active and instrumental to the community since
> > > becoming
> > > > > a
> > > > > > > > > committer.
> > > > > > > > > > > > It's my pleasure to announce that Justine is now a
> > > member of
> > > > > > > Kafka
> > > > > > > > > PMC.
> > > > > > > > > > > >
> > > > > > > > > > > > Congratulations Justine!
> > > > > > > > > > > >
> > > > > > > > > > > > Luke
> > > > > > > > > > > > on behalf of Apache Kafka PMC
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > >
>
>

Re: [ANNOUNCE] New Kafka PMC Member: Justine Olshan

[Satish Duggana]

Congratulations Justine!!

On Sat, 23 Sept 2023 at 15:46, Bill Bejeck  wrote:
>
> Congrats Justine!
>
> -Bill
>
> On Sat, Sep 23, 2023 at 6:23 PM Greg Harris 
> wrote:
>
> > Congratulations Justine!
> >
> > On Sat, Sep 23, 2023 at 5:49 AM Boudjelda Mohamed Said
> >  wrote:
> > >
> > > Congrats Justin !
> > >
> > > On Sat 23 Sep 2023 at 14:44, Randall Hauch  wrote:
> > >
> > > > Congratulations, Justine!
> > > >
> > > > On Sat, Sep 23, 2023 at 4:25 AM Kamal Chandraprakash <
> > > > kamal.chandraprak...@gmail.com> wrote:
> > > >
> > > > > Congrats Justine!
> > > > >
> > > > > On Sat, Sep 23, 2023, 13:28 Divij Vaidya 
> > > > wrote:
> > > > >
> > > > > > Congratulations Justine!
> > > > > >
> > > > > > On Sat 23. Sep 2023 at 07:06, Chris Egerton <
> > fearthecel...@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Congrats Justine!
> > > > > > > On Fri, Sep 22, 2023, 20:47 Guozhang Wang <
> > > > guozhang.wang...@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Congratulations!
> > > > > > > >
> > > > > > > > On Fri, Sep 22, 2023 at 8:44 PM Tzu-Li (Gordon) Tai <
> > > > > > tzuli...@apache.org
> > > > > > > >
> > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > Congratulations Justine!
> > > > > > > > >
> > > > > > > > > On Fri, Sep 22, 2023, 19:25 Philip Nee 
> > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Congrats Justine!
> > > > > > > > > >
> > > > > > > > > > On Fri, Sep 22, 2023 at 7:07 PM Luke Chen <
> > show...@gmail.com>
> > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Hi, Everyone,
> > > > > > > > > > >
> > > > > > > > > > > Justine Olshan has been a Kafka committer since Dec.
> > 2022.
> > > > She
> > > > > > has
> > > > > > > > been
> > > > > > > > > > > very active and instrumental to the community since
> > becoming
> > > > a
> > > > > > > > committer.
> > > > > > > > > > > It's my pleasure to announce that Justine is now a
> > member of
> > > > > > Kafka
> > > > > > > > PMC.
> > > > > > > > > > >
> > > > > > > > > > > Congratulations Justine!
> > > > > > > > > > >
> > > > > > > > > > > Luke
> > > > > > > > > > > on behalf of Apache Kafka PMC
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> >

Re: [ANNOUNCE] New Kafka PMC Member: Justine Olshan

[Bill Bejeck]

Congrats Justine!

-Bill

On Sat, Sep 23, 2023 at 6:23 PM Greg Harris 
wrote:

> Congratulations Justine!
>
> On Sat, Sep 23, 2023 at 5:49 AM Boudjelda Mohamed Said
>  wrote:
> >
> > Congrats Justin !
> >
> > On Sat 23 Sep 2023 at 14:44, Randall Hauch  wrote:
> >
> > > Congratulations, Justine!
> > >
> > > On Sat, Sep 23, 2023 at 4:25 AM Kamal Chandraprakash <
> > > kamal.chandraprak...@gmail.com> wrote:
> > >
> > > > Congrats Justine!
> > > >
> > > > On Sat, Sep 23, 2023, 13:28 Divij Vaidya 
> > > wrote:
> > > >
> > > > > Congratulations Justine!
> > > > >
> > > > > On Sat 23. Sep 2023 at 07:06, Chris Egerton <
> fearthecel...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Congrats Justine!
> > > > > > On Fri, Sep 22, 2023, 20:47 Guozhang Wang <
> > > guozhang.wang...@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Congratulations!
> > > > > > >
> > > > > > > On Fri, Sep 22, 2023 at 8:44 PM Tzu-Li (Gordon) Tai <
> > > > > tzuli...@apache.org
> > > > > > >
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > Congratulations Justine!
> > > > > > > >
> > > > > > > > On Fri, Sep 22, 2023, 19:25 Philip Nee 
> > > > wrote:
> > > > > > > >
> > > > > > > > > Congrats Justine!
> > > > > > > > >
> > > > > > > > > On Fri, Sep 22, 2023 at 7:07 PM Luke Chen <
> show...@gmail.com>
> > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi, Everyone,
> > > > > > > > > >
> > > > > > > > > > Justine Olshan has been a Kafka committer since Dec.
> 2022.
> > > She
> > > > > has
> > > > > > > been
> > > > > > > > > > very active and instrumental to the community since
> becoming
> > > a
> > > > > > > committer.
> > > > > > > > > > It's my pleasure to announce that Justine is now a
> member of
> > > > > Kafka
> > > > > > > PMC.
> > > > > > > > > >
> > > > > > > > > > Congratulations Justine!
> > > > > > > > > >
> > > > > > > > > > Luke
> > > > > > > > > > on behalf of Apache Kafka PMC
> > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
>

Re: [ANNOUNCE] New Kafka PMC Member: Justine Olshan

[Greg Harris]

Congratulations Justine!

On Sat, Sep 23, 2023 at 5:49 AM Boudjelda Mohamed Said
 wrote:
>
> Congrats Justin !
>
> On Sat 23 Sep 2023 at 14:44, Randall Hauch  wrote:
>
> > Congratulations, Justine!
> >
> > On Sat, Sep 23, 2023 at 4:25 AM Kamal Chandraprakash <
> > kamal.chandraprak...@gmail.com> wrote:
> >
> > > Congrats Justine!
> > >
> > > On Sat, Sep 23, 2023, 13:28 Divij Vaidya 
> > wrote:
> > >
> > > > Congratulations Justine!
> > > >
> > > > On Sat 23. Sep 2023 at 07:06, Chris Egerton 
> > > > wrote:
> > > >
> > > > > Congrats Justine!
> > > > > On Fri, Sep 22, 2023, 20:47 Guozhang Wang <
> > guozhang.wang...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Congratulations!
> > > > > >
> > > > > > On Fri, Sep 22, 2023 at 8:44 PM Tzu-Li (Gordon) Tai <
> > > > tzuli...@apache.org
> > > > > >
> > > > > > wrote:
> > > > > > >
> > > > > > > Congratulations Justine!
> > > > > > >
> > > > > > > On Fri, Sep 22, 2023, 19:25 Philip Nee 
> > > wrote:
> > > > > > >
> > > > > > > > Congrats Justine!
> > > > > > > >
> > > > > > > > On Fri, Sep 22, 2023 at 7:07 PM Luke Chen 
> > > > wrote:
> > > > > > > >
> > > > > > > > > Hi, Everyone,
> > > > > > > > >
> > > > > > > > > Justine Olshan has been a Kafka committer since Dec. 2022.
> > She
> > > > has
> > > > > > been
> > > > > > > > > very active and instrumental to the community since becoming
> > a
> > > > > > committer.
> > > > > > > > > It's my pleasure to announce that Justine is now a member of
> > > > Kafka
> > > > > > PMC.
> > > > > > > > >
> > > > > > > > > Congratulations Justine!
> > > > > > > > >
> > > > > > > > > Luke
> > > > > > > > > on behalf of Apache Kafka PMC
> > > > > > > > >
> > > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >

Re: [kafka-clients] [VOTE] 3.6.0 RC1

[Ismael Juma]

Given that this is not a regression and there have been no reports for over
a year, I think it's ok for this to land in 3.6.1.

Ismael

On Sat, Sep 23, 2023 at 9:32 AM Satish Duggana 
wrote:

> Thanks Luke for reporting KRaft issue[1].
>
> I am not sure whether it is a release blocker for 3.6.0. Need input
> from other KRaft experts also to finalize the decision. Even if we
> adopt a fix, do not we need to bake it for some time before it is
> pushed to production to avoid any regressions as this change is in the
> critical paths?
>
> 1. https://issues.apache.org/jira/browse/KAFKA-15489
>
> Thanks,
> Satish.
>
> On Sat, 23 Sept 2023 at 03:08, Luke Chen  wrote:
> >
> > Hi Satish,
> >
> > I found the current KRaft implementation will have "split brain" issue
> when
> > network partition happens, which will cause inconsistent metadata
> returned
> > from the controller.
> > Filed KAFKA-15489 
> for
> > this issue, and PR  is ready
> > for review.
> >
> > Even though this is not a regression issue (this has already existed
> since
> > the 1st release of KRaft feature), I think this is an important issue
> since
> > KRaft is announced production ready.
> > Not sure what other people's thoughts are.
> >
> > Thank you.
> > Luke
> >
> > On Thu, Sep 21, 2023 at 6:33 PM Josep Prat 
> > wrote:
> >
> > > Hi Satish,
> > >
> > > I ran the following validation steps:
> > > - Built from source with Java 11 and Scala 2.13
> > > - Verified Signatures and hashes of the artifacts generated
> > > - Navigated through Javadoc including links to JDK classes
> > > - Run the unit tests
> > > - Run integration tests
> > > - Run the quickstart in KRaft and Zookeeper mode
> > >
> > >
> > > I +1 this release (non-binding)
> > >
> > > Thanks for your efforts!
> > >
> > > On Thu, Sep 21, 2023 at 2:59 AM Satish Duggana <
> satish.dugg...@gmail.com>
> > > wrote:
> > >
> > > > Thanks Greg for verifying the release including the earlier
> > > > blocker(KAFKA-15473) verification.
> > > >
> > > > ~Satish.
> > > >
> > > > On Wed, 20 Sept 2023 at 22:30, Greg Harris
>  > > >
> > > > wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > I verified the functionality of KIP-898 and the recent fix for
> > > > > KAFKA-15473 with the following steps:
> > > > >
> > > > > 1. I started a 3.5.1 broker, and a 3.5.1 worker with most (>400)
> > > > > publicly available plugins installed
> > > > > 2. I captured the output of /connector-plugins
> > > > > 3. I upgraded the worker to 3.6.0-rc1
> > > > > 4. I captured the output of /connector-plugins with various
> settings
> > > > > of plugin.discovery
> > > > > 5. I ran the migration script to add manifests to my plugins
> > > > > 6. I captured the output of /connector-plugins with various
> settings
> > > > > of plugin.discovery
> > > > > 7. I downgraded the worker to 3.5.1
> > > > > 8. I diffed the output of /connector-plugins across the different
> > > > > cases and observed the expected changes.
> > > > > a. When plugins are migrated for 3.6.0, all modes produce
> identical
> > > > > results.
> > > > > b. When plugins are not migrated for 3.6.0, only_scan and
> > > > > hybrid_warn produce identical results, hybrid_fail crashes, and
> > > > > service_load is missing plugins
> > > > > c. When upgrading from 3.5.1 I see that plugins with invalid
> > > > > constructors are hidden, AK plugins now have versions,
> multi-interface
> > > > > plugins now show each interface type, and plugins using
> AppInfoParser
> > > > > change versions.
> > > > > d. The startup logs now include descriptive errors for invalid
> > > > > plugins that otherwise would have been thrown at runtime
> > > > > d. The fix for KAFKA-15473 prevents duplicates
> > > > > e. The output for 3.5.1 after downgrading is identical to
> before.
> > > > >
> > > > > +1 (non-binding)
> > > > >
> > > > > Thanks Satish for running the release!
> > > > >
> > > > > On Wed, Sep 20, 2023 at 8:36 AM Divij Vaidya 
> > > wrote:
> > > > > >
> > > > > > Hey Satish
> > > > > >
> > > > > > My comments about documentation misses from RC0 vote thread [1]
> are
> > > > > > still not addressed (such as missing metric documentation,
> formatting
> > > > > > problems etc). Could you please mention why we shouldn't consider
> > > them
> > > > > > as blockers to make RC1 as the final release?
> > > > > >
> > > > > > [1]
> https://lists.apache.org/thread/cokoxzd0jtgjtrlxoq7kkzmvpm75381t
> > > > > >
> > > > > > On Wed, Sep 20, 2023 at 4:53 PM Satish Duggana <
> > > > satish.dugg...@gmail.com>
> > > > > wrote:
> > > > > > >
> > > > > > > Hello Kafka users, developers and client-developers,
> > > > > > >
> > > > > > > This is the second candidate for the release of Apache Kafka
> 3.6.0.
> > > > > Some of the major features include:
> > > > > > >
> > > > > > > * KIP-405 : Kafka Tiered Storage
> > > > > > > * KIP-868 : KRaft Metadata Transactions
> > > > > > > * KIP-

Re: [kafka-clients] [VOTE] 3.6.0 RC1

[Satish Duggana]

Thanks Luke for reporting KRaft issue[1].

I am not sure whether it is a release blocker for 3.6.0. Need input
from other KRaft experts also to finalize the decision. Even if we
adopt a fix, do not we need to bake it for some time before it is
pushed to production to avoid any regressions as this change is in the
critical paths?

1. https://issues.apache.org/jira/browse/KAFKA-15489

Thanks,
Satish.

On Sat, 23 Sept 2023 at 03:08, Luke Chen  wrote:
>
> Hi Satish,
>
> I found the current KRaft implementation will have "split brain" issue when
> network partition happens, which will cause inconsistent metadata returned
> from the controller.
> Filed KAFKA-15489  for
> this issue, and PR  is ready
> for review.
>
> Even though this is not a regression issue (this has already existed since
> the 1st release of KRaft feature), I think this is an important issue since
> KRaft is announced production ready.
> Not sure what other people's thoughts are.
>
> Thank you.
> Luke
>
> On Thu, Sep 21, 2023 at 6:33 PM Josep Prat 
> wrote:
>
> > Hi Satish,
> >
> > I ran the following validation steps:
> > - Built from source with Java 11 and Scala 2.13
> > - Verified Signatures and hashes of the artifacts generated
> > - Navigated through Javadoc including links to JDK classes
> > - Run the unit tests
> > - Run integration tests
> > - Run the quickstart in KRaft and Zookeeper mode
> >
> >
> > I +1 this release (non-binding)
> >
> > Thanks for your efforts!
> >
> > On Thu, Sep 21, 2023 at 2:59 AM Satish Duggana 
> > wrote:
> >
> > > Thanks Greg for verifying the release including the earlier
> > > blocker(KAFKA-15473) verification.
> > >
> > > ~Satish.
> > >
> > > On Wed, 20 Sept 2023 at 22:30, Greg Harris  > >
> > > wrote:
> > >
> > > > Hi all,
> > > >
> > > > I verified the functionality of KIP-898 and the recent fix for
> > > > KAFKA-15473 with the following steps:
> > > >
> > > > 1. I started a 3.5.1 broker, and a 3.5.1 worker with most (>400)
> > > > publicly available plugins installed
> > > > 2. I captured the output of /connector-plugins
> > > > 3. I upgraded the worker to 3.6.0-rc1
> > > > 4. I captured the output of /connector-plugins with various settings
> > > > of plugin.discovery
> > > > 5. I ran the migration script to add manifests to my plugins
> > > > 6. I captured the output of /connector-plugins with various settings
> > > > of plugin.discovery
> > > > 7. I downgraded the worker to 3.5.1
> > > > 8. I diffed the output of /connector-plugins across the different
> > > > cases and observed the expected changes.
> > > > a. When plugins are migrated for 3.6.0, all modes produce identical
> > > > results.
> > > > b. When plugins are not migrated for 3.6.0, only_scan and
> > > > hybrid_warn produce identical results, hybrid_fail crashes, and
> > > > service_load is missing plugins
> > > > c. When upgrading from 3.5.1 I see that plugins with invalid
> > > > constructors are hidden, AK plugins now have versions, multi-interface
> > > > plugins now show each interface type, and plugins using AppInfoParser
> > > > change versions.
> > > > d. The startup logs now include descriptive errors for invalid
> > > > plugins that otherwise would have been thrown at runtime
> > > > d. The fix for KAFKA-15473 prevents duplicates
> > > > e. The output for 3.5.1 after downgrading is identical to before.
> > > >
> > > > +1 (non-binding)
> > > >
> > > > Thanks Satish for running the release!
> > > >
> > > > On Wed, Sep 20, 2023 at 8:36 AM Divij Vaidya 
> > wrote:
> > > > >
> > > > > Hey Satish
> > > > >
> > > > > My comments about documentation misses from RC0 vote thread [1] are
> > > > > still not addressed (such as missing metric documentation, formatting
> > > > > problems etc). Could you please mention why we shouldn't consider
> > them
> > > > > as blockers to make RC1 as the final release?
> > > > >
> > > > > [1] https://lists.apache.org/thread/cokoxzd0jtgjtrlxoq7kkzmvpm75381t
> > > > >
> > > > > On Wed, Sep 20, 2023 at 4:53 PM Satish Duggana <
> > > satish.dugg...@gmail.com>
> > > > wrote:
> > > > > >
> > > > > > Hello Kafka users, developers and client-developers,
> > > > > >
> > > > > > This is the second candidate for the release of Apache Kafka 3.6.0.
> > > > Some of the major features include:
> > > > > >
> > > > > > * KIP-405 : Kafka Tiered Storage
> > > > > > * KIP-868 : KRaft Metadata Transactions
> > > > > > * KIP-875: First-class offsets support in Kafka Connect
> > > > > > * KIP-898: Modernize Connect plugin discovery
> > > > > > * KIP-938: Add more metrics for measuring KRaft performance
> > > > > > * KIP-902: Upgrade Zookeeper to 3.8.1
> > > > > > * KIP-917: Additional custom metadata for remote log segment
> > > > > >
> > > > > > Release notes for the 3.6.0 release:
> > > > > >
> > https://home.apache.org/~satishd/kafka-3.6.0-rc1/RELEASE_NOTES.html
> > > > > >
> > > > > > *** Please download, 

[DISCUSS] KIP-983: Full speed async processing during rebalance

[Erik van Oosten]

Hi all,

I would like to start the discussion on KIP-983: Full speed async 
processing during rebalance [1].


The idea is that we can prevent the drop in throughput during a 
cooperative rebalance.


I am curious to your ideas and comments.

Kind regards,
    Erik.

[1] 
https://cwiki.apache.org/confluence/display/KAFKA/KIP-983%3A+Full+speed+async+processing+during+rebalance



--
Erik van Oosten
e.vanoos...@grons.nl
https://day-to-day-stuff.blogspot.com

Re: [VOTE] KIP-858: Handle JBOD broker disk failure in KRaft

[Igor Soarez]

Hi everyone,

I believe we can close this voting thread now, as there
were three +1 binding votes from Ziming, Mickael and Ron.
With that, this vote passes.

Thanks to everyone who participated in reviewing,
and/or taking the time to vote on this KIP!

Best,

--
Igor

Re: [VOTE] KIP-858: Handle JBOD broker disk failure in KRaft

[Igor Soarez]

Hi Ron,

Thanks for pointing this out.

I was assuming this case was already handled in current KRaft operation
with a single log directory — I wouldn't expect a broker restarting
with an empty disk to cause data loss in a current KRaft system.

But your question made me go look again, so here's my understanding:

When unclean leader election (ULE) is disabled (by default),
a broker can only be elected as a leader if both of these are true:
* It is part of the ISR - see PartitionChangeBuilder.isValidNewLeader(r)
* It is either
  + Not fenced and not in a controlled shutdown; or
  + Currently being unfenced

Unless of course unclean leader election is enabled, in which case
yes it looks like you could have full data loss. But I don't think
we want to address this case.

If the broker is coming back with an empty disk, it would still
need to join the ISR to become the leader.
So as far as I can tell, this doesn't seem to be a problem.

If we do always add the assignment, even for single log dir uses,
there could potentially be some smarter and safer choices we can do
in the broker, but AFAICT this doesn't seem to be a hard requirement
right now.

And as Colin pointed out before, addding the assignment even when
only a single dir is configured may unecessarily penalize non JBOD
usecases.

Does this make sense? Did I understand your point correctly? WDYT?

--
Igor

On Fri, Sep 22, 2023, at 2:44 PM, Ron Dagostino wrote:
> Hi Igor.  Someone just asked about the case where a broker with a
> single log directory restarts with a blank disk.  I looked at the
> "Metadata caching" section, and I don't think it covers it as
> currently written.  The PartitionRecord will not have an explicit UUID
> for brokers that have just a single log directory (the idea was to
> save space), but the UUID is inferred to be the UUID associated with
> the single log directory that the broker had registered.  Assume a
> broker restarts with an empty disk.  That disk will get a new UUID,
> and upon broker registration the controller will see the UUID mismatch
> between what the broker is presenting now and what it had presented
> the last time it registered.  So we need to deal with this possibility
> even for the case where a broker has a single log directory.  WDYT?
> 
> Ron
> 
> On Tue, Sep 19, 2023 at 10:04 AM Ron Dagostino  wrote:
> >
> > Ok, great, that makes sense, Igor.  Thanks.  +1 (binding) on the KIP from 
> > me.
> >
> > Ron
> >
> > > On Sep 13, 2023, at 11:58 AM, Igor Soarez  
> > > wrote:
> > >
> > > Hi Ron,
> > >
> > > Thanks for drilling down on this. I think the KIP isn't really clear here,
> > > and the metadata caching section you quoted needs clarification.
> > >
> > > The "hosting broker's latest registration" refers to the previous,
> > > not the current registration. The registrations are only compared by
> > > the controller, when handling the broker registration request.
> > >
> > > Suppose broker b1 hosts two partitions, t-1 and t-2, in two
> > > directories, d1 and d2. The broker is registered, and the
> > > metadata correlates the replicas to their respective directories.
> > > i.e. OnlineLogDirs=[d1,d2] and OfflineLogDirs=false
> > >
> > > The broker is then reconfigured to remove t-2 from log.dirs, and at 
> > > startup,
> > > the registration request shows OnlineLogDirs=[d1] and 
> > > OfflineLogDirs=false.
> > > The previous registration will only be replaced after a new successful
> > > registration, regardless of how quickly or how often b1 restarts.
> > > The controller compares the previous registration, and notices
> > > that one of the directories has been removed.
> > > So for any replica hosted in the broker that is assigned to that
> > > missing log directory, a logical metadata update takes place
> > > that assigned them to Uuid.OfflineDir, so Assignment.Directory
> > > is updated for t-2. This value is indicates that the replica
> > > is offline — I have updated the section you quoted to address this.
> > >
> > > Once the broker catches up with metadata, it will select the only
> > > configured log directory — d1 — for any partitions assigned to
> > > Uuid.OfflineDir, and update the assignment.
> > >
> > > Best,
> > >
> > > --
> > > Igor
> > >
> > >
> > >
> 

Re: [ANNOUNCE] New Kafka PMC Member: Justine Olshan

[Boudjelda Mohamed Said]

Congrats Justin !

On Sat 23 Sep 2023 at 14:44, Randall Hauch  wrote:

> Congratulations, Justine!
>
> On Sat, Sep 23, 2023 at 4:25 AM Kamal Chandraprakash <
> kamal.chandraprak...@gmail.com> wrote:
>
> > Congrats Justine!
> >
> > On Sat, Sep 23, 2023, 13:28 Divij Vaidya 
> wrote:
> >
> > > Congratulations Justine!
> > >
> > > On Sat 23. Sep 2023 at 07:06, Chris Egerton 
> > > wrote:
> > >
> > > > Congrats Justine!
> > > > On Fri, Sep 22, 2023, 20:47 Guozhang Wang <
> guozhang.wang...@gmail.com>
> > > > wrote:
> > > >
> > > > > Congratulations!
> > > > >
> > > > > On Fri, Sep 22, 2023 at 8:44 PM Tzu-Li (Gordon) Tai <
> > > tzuli...@apache.org
> > > > >
> > > > > wrote:
> > > > > >
> > > > > > Congratulations Justine!
> > > > > >
> > > > > > On Fri, Sep 22, 2023, 19:25 Philip Nee 
> > wrote:
> > > > > >
> > > > > > > Congrats Justine!
> > > > > > >
> > > > > > > On Fri, Sep 22, 2023 at 7:07 PM Luke Chen 
> > > wrote:
> > > > > > >
> > > > > > > > Hi, Everyone,
> > > > > > > >
> > > > > > > > Justine Olshan has been a Kafka committer since Dec. 2022.
> She
> > > has
> > > > > been
> > > > > > > > very active and instrumental to the community since becoming
> a
> > > > > committer.
> > > > > > > > It's my pleasure to announce that Justine is now a member of
> > > Kafka
> > > > > PMC.
> > > > > > > >
> > > > > > > > Congratulations Justine!
> > > > > > > >
> > > > > > > > Luke
> > > > > > > > on behalf of Apache Kafka PMC
> > > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [ANNOUNCE] New Kafka PMC Member: Justine Olshan

[Randall Hauch]

Congratulations, Justine!

On Sat, Sep 23, 2023 at 4:25 AM Kamal Chandraprakash <
kamal.chandraprak...@gmail.com> wrote:

> Congrats Justine!
>
> On Sat, Sep 23, 2023, 13:28 Divij Vaidya  wrote:
>
> > Congratulations Justine!
> >
> > On Sat 23. Sep 2023 at 07:06, Chris Egerton 
> > wrote:
> >
> > > Congrats Justine!
> > > On Fri, Sep 22, 2023, 20:47 Guozhang Wang 
> > > wrote:
> > >
> > > > Congratulations!
> > > >
> > > > On Fri, Sep 22, 2023 at 8:44 PM Tzu-Li (Gordon) Tai <
> > tzuli...@apache.org
> > > >
> > > > wrote:
> > > > >
> > > > > Congratulations Justine!
> > > > >
> > > > > On Fri, Sep 22, 2023, 19:25 Philip Nee 
> wrote:
> > > > >
> > > > > > Congrats Justine!
> > > > > >
> > > > > > On Fri, Sep 22, 2023 at 7:07 PM Luke Chen 
> > wrote:
> > > > > >
> > > > > > > Hi, Everyone,
> > > > > > >
> > > > > > > Justine Olshan has been a Kafka committer since Dec. 2022. She
> > has
> > > > been
> > > > > > > very active and instrumental to the community since becoming a
> > > > committer.
> > > > > > > It's my pleasure to announce that Justine is now a member of
> > Kafka
> > > > PMC.
> > > > > > >
> > > > > > > Congratulations Justine!
> > > > > > >
> > > > > > > Luke
> > > > > > > on behalf of Apache Kafka PMC
> > > > > > >
> > > > > >
> > > >
> > >
> >
>

[jira] [Created] (KAFKA-15490) Invalid path provided to the log failure channel upon I/O error when writing broker metadata checkpoint

[Alexandre Dupriez (Jira)]

Alexandre Dupriez created KAFKA-15490:
-

 Summary: Invalid path provided to the log failure channel upon I/O 
error when writing broker metadata checkpoint
 Key: KAFKA-15490
 URL: https://issues.apache.org/jira/browse/KAFKA-15490
 Project: Kafka
  Issue Type: Bug
  Components: core
Reporter: Alexandre Dupriez


There is a small bug/typo in the handling of I/O error when writing broker 
metadata checkpoint in {{{}KafkaServer{}}}. The path provided to the log dir 
failure channel is the full path of the checkpoint file whereas only the log 
directory is expected 
([source|https://github.com/apache/kafka/blob/3.4/core/src/main/scala/kafka/server/KafkaServer.scala#L958C8-L961C8]).

 
{code:java}
case e: IOException =>
          val dirPath = checkpoint.file.getAbsolutePath
          logDirFailureChannel.maybeAddOfflineLogDir(dirPath, s"Error while 
writing meta.properties to $dirPath", e){code}
 

As a result, after an {{IOException}} is captured and enqueued in the log dir 
failure channel 

{{}}
{code:java}

{code}
{{[2023-09-22 17:07:32,052] ERROR Error while writing meta.properties to 
/meta.properties (kafka.server.LogDirFailureChannel) 
java.io.IOException}}

 

The log dir failure handler cannot lookup the log directory:

{{}}
{code:java}

{code}
{{[2023-09-22 17:07:32,053] ERROR [LogDirFailureHandler]: Error due to 
(kafka.server.ReplicaManager$LogDirFailureHandler) 
org.apache.kafka.common.errors.LogDirNotFoundException: Log dir 
/meta.properties is not found in the config.}}

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [kafka-clients] [VOTE] 3.6.0 RC1

[Luke Chen]

Hi Satish,

I found the current KRaft implementation will have "split brain" issue when
network partition happens, which will cause inconsistent metadata returned
from the controller.
Filed KAFKA-15489  for
this issue, and PR  is ready
for review.

Even though this is not a regression issue (this has already existed since
the 1st release of KRaft feature), I think this is an important issue since
KRaft is announced production ready.
Not sure what other people's thoughts are.

Thank you.
Luke

On Thu, Sep 21, 2023 at 6:33 PM Josep Prat 
wrote:

> Hi Satish,
>
> I ran the following validation steps:
> - Built from source with Java 11 and Scala 2.13
> - Verified Signatures and hashes of the artifacts generated
> - Navigated through Javadoc including links to JDK classes
> - Run the unit tests
> - Run integration tests
> - Run the quickstart in KRaft and Zookeeper mode
>
>
> I +1 this release (non-binding)
>
> Thanks for your efforts!
>
> On Thu, Sep 21, 2023 at 2:59 AM Satish Duggana 
> wrote:
>
> > Thanks Greg for verifying the release including the earlier
> > blocker(KAFKA-15473) verification.
> >
> > ~Satish.
> >
> > On Wed, 20 Sept 2023 at 22:30, Greg Harris  >
> > wrote:
> >
> > > Hi all,
> > >
> > > I verified the functionality of KIP-898 and the recent fix for
> > > KAFKA-15473 with the following steps:
> > >
> > > 1. I started a 3.5.1 broker, and a 3.5.1 worker with most (>400)
> > > publicly available plugins installed
> > > 2. I captured the output of /connector-plugins
> > > 3. I upgraded the worker to 3.6.0-rc1
> > > 4. I captured the output of /connector-plugins with various settings
> > > of plugin.discovery
> > > 5. I ran the migration script to add manifests to my plugins
> > > 6. I captured the output of /connector-plugins with various settings
> > > of plugin.discovery
> > > 7. I downgraded the worker to 3.5.1
> > > 8. I diffed the output of /connector-plugins across the different
> > > cases and observed the expected changes.
> > > a. When plugins are migrated for 3.6.0, all modes produce identical
> > > results.
> > > b. When plugins are not migrated for 3.6.0, only_scan and
> > > hybrid_warn produce identical results, hybrid_fail crashes, and
> > > service_load is missing plugins
> > > c. When upgrading from 3.5.1 I see that plugins with invalid
> > > constructors are hidden, AK plugins now have versions, multi-interface
> > > plugins now show each interface type, and plugins using AppInfoParser
> > > change versions.
> > > d. The startup logs now include descriptive errors for invalid
> > > plugins that otherwise would have been thrown at runtime
> > > d. The fix for KAFKA-15473 prevents duplicates
> > > e. The output for 3.5.1 after downgrading is identical to before.
> > >
> > > +1 (non-binding)
> > >
> > > Thanks Satish for running the release!
> > >
> > > On Wed, Sep 20, 2023 at 8:36 AM Divij Vaidya 
> wrote:
> > > >
> > > > Hey Satish
> > > >
> > > > My comments about documentation misses from RC0 vote thread [1] are
> > > > still not addressed (such as missing metric documentation, formatting
> > > > problems etc). Could you please mention why we shouldn't consider
> them
> > > > as blockers to make RC1 as the final release?
> > > >
> > > > [1] https://lists.apache.org/thread/cokoxzd0jtgjtrlxoq7kkzmvpm75381t
> > > >
> > > > On Wed, Sep 20, 2023 at 4:53 PM Satish Duggana <
> > satish.dugg...@gmail.com>
> > > wrote:
> > > > >
> > > > > Hello Kafka users, developers and client-developers,
> > > > >
> > > > > This is the second candidate for the release of Apache Kafka 3.6.0.
> > > Some of the major features include:
> > > > >
> > > > > * KIP-405 : Kafka Tiered Storage
> > > > > * KIP-868 : KRaft Metadata Transactions
> > > > > * KIP-875: First-class offsets support in Kafka Connect
> > > > > * KIP-898: Modernize Connect plugin discovery
> > > > > * KIP-938: Add more metrics for measuring KRaft performance
> > > > > * KIP-902: Upgrade Zookeeper to 3.8.1
> > > > > * KIP-917: Additional custom metadata for remote log segment
> > > > >
> > > > > Release notes for the 3.6.0 release:
> > > > >
> https://home.apache.org/~satishd/kafka-3.6.0-rc1/RELEASE_NOTES.html
> > > > >
> > > > > *** Please download, test and vote by Saturday, September 23, 8am
> PT
> > > > >
> > > > > Kafka's KEYS file containing PGP keys we use to sign the release:
> > > > > https://kafka.apache.org/KEYS
> > > > >
> > > > > * Release artifacts to be voted upon (source and binary):
> > > > > https://home.apache.org/~satishd/kafka-3.6.0-rc1/
> > > > >
> > > > > * Maven artifacts to be voted upon:
> > > > >
> > https://repository.apache.org/content/groups/staging/org/apache/kafka/
> > > > >
> > > > > * Javadoc:
> > > > > https://home.apache.org/~satishd/kafka-3.6.0-rc1/javadoc/
> > > > >
> > > > > * Tag to be voted upon (off 3.6 branch) is the 3.6.0 tag:
> > > > > https://github.com/apache/kafka/releases/

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #2225

[Apache Jenkins Server]

See 

Re: [ANNOUNCE] New Kafka PMC Member: Justine Olshan

[Kamal Chandraprakash]

Congrats Justine!

On Sat, Sep 23, 2023, 13:28 Divij Vaidya  wrote:

> Congratulations Justine!
>
> On Sat 23. Sep 2023 at 07:06, Chris Egerton 
> wrote:
>
> > Congrats Justine!
> > On Fri, Sep 22, 2023, 20:47 Guozhang Wang 
> > wrote:
> >
> > > Congratulations!
> > >
> > > On Fri, Sep 22, 2023 at 8:44 PM Tzu-Li (Gordon) Tai <
> tzuli...@apache.org
> > >
> > > wrote:
> > > >
> > > > Congratulations Justine!
> > > >
> > > > On Fri, Sep 22, 2023, 19:25 Philip Nee  wrote:
> > > >
> > > > > Congrats Justine!
> > > > >
> > > > > On Fri, Sep 22, 2023 at 7:07 PM Luke Chen 
> wrote:
> > > > >
> > > > > > Hi, Everyone,
> > > > > >
> > > > > > Justine Olshan has been a Kafka committer since Dec. 2022. She
> has
> > > been
> > > > > > very active and instrumental to the community since becoming a
> > > committer.
> > > > > > It's my pleasure to announce that Justine is now a member of
> Kafka
> > > PMC.
> > > > > >
> > > > > > Congratulations Justine!
> > > > > >
> > > > > > Luke
> > > > > > on behalf of Apache Kafka PMC
> > > > > >
> > > > >
> > >
> >
>

Re: [ANNOUNCE] New Kafka PMC Member: Justine Olshan

[Divij Vaidya]

Congratulations Justine!

On Sat 23. Sep 2023 at 07:06, Chris Egerton  wrote:

> Congrats Justine!
> On Fri, Sep 22, 2023, 20:47 Guozhang Wang 
> wrote:
>
> > Congratulations!
> >
> > On Fri, Sep 22, 2023 at 8:44 PM Tzu-Li (Gordon) Tai  >
> > wrote:
> > >
> > > Congratulations Justine!
> > >
> > > On Fri, Sep 22, 2023, 19:25 Philip Nee  wrote:
> > >
> > > > Congrats Justine!
> > > >
> > > > On Fri, Sep 22, 2023 at 7:07 PM Luke Chen  wrote:
> > > >
> > > > > Hi, Everyone,
> > > > >
> > > > > Justine Olshan has been a Kafka committer since Dec. 2022. She has
> > been
> > > > > very active and instrumental to the community since becoming a
> > committer.
> > > > > It's my pleasure to announce that Justine is now a member of Kafka
> > PMC.
> > > > >
> > > > > Congratulations Justine!
> > > > >
> > > > > Luke
> > > > > on behalf of Apache Kafka PMC
> > > > >
> > > >
> >
>

[jira] [Created] (KAFKA-15489) split brain in KRaft cluster

[Luke Chen (Jira)]

Luke Chen created KAFKA-15489:
-

 Summary: split brain in KRaft cluster 
 Key: KAFKA-15489
 URL: https://issues.apache.org/jira/browse/KAFKA-15489
 Project: Kafka
  Issue Type: Bug
  Components: kraft
Affects Versions: 3.5.1
Reporter: Luke Chen
Assignee: Luke Chen


I found in the current KRaft implementation, when network partition happened 
between the current controller leader and the other controller nodes, the 
"split brain" issue will happen. It causes 2 leaders will exist in the 
controller cluster, and 2 inconsistent sets of metadata will return to the 
clients.

 

*Root cause*
In 
[KIP-595|https://cwiki.apache.org/confluence/display/KAFKA/KIP-595%3A+A+Raft+Protocol+for+the+Metadata+Quorum#KIP595:ARaftProtocolfortheMetadataQuorum-Vote],
 we said A voter will begin a new election under three conditions:

1. If it fails to receive a FetchResponse from the current leader before 
expiration of quorum.fetch.timeout.ms
2. If it receives a EndQuorumEpoch request from the current leader
3. If it fails to receive a majority of votes before expiration of 
quorum.election.timeout.ms after declaring itself a candidate.

And that's exactly what the current KRaft's implementation.

 

However, when the leader is isolated from the network partition, there's no way 
for it to resign from the leadership and start a new election. So the leader 
will always be the leader even though all other nodes are down. And this makes 
the split brain issue possible.

When reading further in the KIP-595, I found we indeed considered this 
situation and have solution for that. in [this 
section|https://cwiki.apache.org/confluence/display/KAFKA/KIP-595%3A+A+Raft+Protocol+for+the+Metadata+Quorum#KIP595:ARaftProtocolfortheMetadataQuorum-LeaderProgressTimeout],
 it said:
{quote}In the pull-based model, however, say a new leader has been elected with 
a new epoch and everyone has learned about it except the old leader (e.g. that 
leader was not in the voters anymore and hence not receiving the 
BeginQuorumEpoch as well), then that old leader would not be notified by anyone 
about the new leader / epoch and become a pure "zombie leader", as there is no 
regular heartbeats being pushed from leader to the follower. This could lead to 
stale information being served to the observers and clients inside the cluster.
{quote}
{quote}To resolve this issue, we will piggy-back on the 
"quorum.fetch.timeout.ms" config, such that if the leader did not receive Fetch 
requests from a majority of the quorum for that amount of time, it would begin 
a new election and start sending VoteRequest to voter nodes in the cluster to 
understand the latest quorum. If it couldn't connect to any known voter, the 
old leader shall keep starting new elections and bump the epoch.
{quote}
 

But we missed this implementation in current KRaft.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)