Hi, 

Hope you all are safe and well!

Before I start with the issue I am trying to solve, let me give you few details 
- 

OS: Ubuntu 18.0.4
ZK version: 3.5.9
Kafka version : confluent-kaka 6.2.4-1

So, we have multiple 3 node setup of Zookeeper nodes to manage multiple Kafka 
Cluster and we ran into an issue when the AZ in which both the ZK leader as 
well as the Kafka controller broker had an outage. The issue is that the ZK 
followers took approximately 16 mins to realise they need to undergo an 
election and form a Zookeeper cluster with the remaining 2 nodes in the 
cluster. This also resulted in an outage to the Kafka users/clients as it was 
also down for the whole 16mins.

We would like to - 
1. Avoid this issue either by moving the leadership to one of the followers 
2. Know if there is any parameter setting to help the ZK cluster to identify a 
new leader by itself

Note: Please let me know if I need to share any Zookeeper / Kafka logs or 
anything else to look further into this.

I read the https://zookeeper.apache.org/doc/r3.5.3-beta/zookeeperReconfig.html 
<https://zookeeper.apache.org/doc/r3.5.3-beta/zookeeperReconfig.html>. Though 
it sounds like it might serve the purpose 1 above, I am not sure if that is the 
way to go here for my requirement as it more or less speaks about dynamically 
adding/removing zk nodes into an existing cluster.

Hoping to find a fix for this issue at the earliest. Thanking you in advance!

Regards,
Rijo Roy

Reply via email to