Hi Dima, Before v3.9.0, brokers in KRaft mode didn't honor `unclean.leader.election.enable` config. So, the static config and dynamic config didn't work. The only thing works before v3.9.0 is using he admin API or cli to run `kafka-leader-election.sh` with `unclean` option.
Thanks. Luke On Thu, Aug 29, 2024 at 10:39 AM Dima Brodsky <ddbrod...@gmail.com> wrote: > Thanks Luke, > > I meant in pre 3.9.x, like 3.6.x or 3.7.x branch for example. > > ttyl > Dima > > > On Wed, Aug 28, 2024 at 7:36 PM Luke Chen <show...@gmail.com> wrote: > > > Hi Dima, > > > > > I assume that unclean leader election exists as a static config in > kafka > > kraft, but is still unavailable dynamically? Or is it unavailable in > > general? > > > > Let me make it clear. In v3.9.0, the `unclean.leader.election.enable` in > > KRaft will work, either static config or dynamic config. > > It's just the behavior changes when dynamically enabling the > > `unclean.leader.election.enable`, which will not trigger unclean leader > > election immediately after config change. The detailed behavior change is > > as described above. > > > > Let me know if it's still not clear. > > > > Thanks. > > Luke > > > > > > On Thu, Aug 29, 2024 at 1:58 AM Dima Brodsky <ddbrod...@gmail.com> > wrote: > > > > > Hi Luke, > > > > > > I assume that unclean leader election exists as a static config in > kafka > > > kraft, but is still unavailable dynamically? Or is it unavailable in > > > general? > > > > > > Thanks! > > > ttyl > > > Dima > > > > > > > > > On Wed, Aug 28, 2024 at 3:45 AM Luke Chen <show...@gmail.com> wrote: > > > > > > > Hi all, > > > > > > > > In v3.9.0, we're planning to support the > > `unclean.leader.election.enable` > > > > config in KRaft. Currently, the implementation PR > > > > <https://github.com/apache/kafka/pull/16866> is under review now. > But > > > I'd > > > > like to raise the discussion to let you know there will be one > behavior > > > > change when enabling `unclean.leader.election.enable` config > > dynamically > > > in > > > > KRaft. > > > > > > > > scenario: > > > > 1. broker [0, 1] config are all set > > > `unclean.leader.election.enable=false`. > > > > 2. topic A is created with 1 partition, 2 replication factors and > > > > `unclean.leader.election.enable=false`. Leader of the partition 0 is > 0. > > > ISR > > > > is [0]. > > > > 3. broker 0 is down, which leaves ISR to empty [ ], and leader none. > > > > 4. dynamically alter config using admin API to set > > > > `unclean.leader.election.enable=true`, no matter it changes to the > > topic > > > A, > > > > or to broker 1, or default broker. > > > > > > > > In ZK mode, with this scenario, it'll trigger unclean leader election > > > > immediately, and broker 1 will be elected as the leader, which brings > > the > > > > availability of the partition back. > > > > In KRaft mode, with this scenario, the unclean leader election will > be > > > > triggered in an unclean election thread, which is scheduled every 5 > > mins > > > by > > > > default. That is, this partition might still be unavailable even if > it > > > > enables `unclean.leader.election.enable`. If users want to elect a > > leader > > > > for this partition, run `kafka-leader-election.sh` with `unclean` > > option > > > to > > > > trigger the unclean leader election immediately. > > > > > > > > The main reason we didn't implement what ZK did is because of this > > > > complication > > > > <https://github.com/apache/kafka/pull/16284#discussion_r1635410206> > we > > > > faced. > > > > > > > > Other behaviors keep the same. Ex: When the topic config, or broker > > > config, > > > > or default broker config set `unclean.leader.election.enable=true`, > and > > > > later, a partition has leader down with empty ISR [ ], and having > some > > > > alive non-ISR replicas, in this case, the unclean leader election > will > > be > > > > triggered, as long as the config was set earlier. > > > > > > > > But since the PR is still under review, any comments or thoughts are > > > > welcomed. > > > > > > > > Thank you. > > > > Luke > > > > > > > > > > > > > -- > > > ddbrod...@gmail.com > > > > > > "The price of reliability is the pursuit of the utmost simplicity. > > > It is a price which the very rich find the most hard to pay." > > > (Sir > > > Antony Hoare, 1980) > > > > > > > > -- > ddbrod...@gmail.com > > "The price of reliability is the pursuit of the utmost simplicity. > It is a price which the very rich find the most hard to pay." > (Sir > Antony Hoare, 1980) >
