Sravani created KAFKA-19785:
-------------------------------
Summary: Two Kafka brokers were not active in 3 node cluster setup
Key: KAFKA-19785
URL: https://issues.apache.org/jira/browse/KAFKA-19785
Project: Kafka
Issue Type: Bug
Affects Versions: 4.0.0
Reporter: Sravani
Hi Team,
We were facing kafka issue where two of the kafka brokers were fenced and Kafka
was not able to process messages. We are using Kafka 4.0.0 version. Below are
the errors.
Sep 22 09:41:42 s99ossv152 kafka[42245]: [2025-09-22 07:41:42,419] ERROR
Encountered fatal fault: Unexpected error in raft IO thread
(org.apache.kafka.server.fault.ProcessTerminatingFaultHandler)
Sep 22 09:41:42 s99ossv152 kafka[42245]: java.lang.IllegalStateException:
Received request or response with leader OptionalInt[3] and epoch 55 *which is
inconsistent with current leader* OptionalInt.empty and epoch 55
Sep 22 09:41:42 s99ossv152 kafka[42245]: #011at
org.apache.kafka.raft.KafkaRaftClient.maybeTransition(KafkaRaftClient.java:2528)
~[kafka-raft-4.0.0.jar:?]
Sep 22 09:41:42 s99ossv152 kafka[42245]: #011at
org.apache.kafka.raft.KafkaRaftClient.maybeHandleCommonResponse(KafkaRaftClient.java:2484)
~[kafka-raft-4.0.0.jar:?]
Sep 22 09:41:42 s99ossv152 kafka[42245]: #011at
org.apache.kafka.raft.KafkaRaftClient.handleFetchResponse(KafkaRaftClient.java:1707)
~[kafka-raft-4.0.0.jar:?]
Sep 22 09:41:42 s99ossv152 kafka[42245]: #011at
org.apache.kafka.raft.KafkaRaftClient.handleResponse(KafkaRaftClient.java:2568)
~[kafka-raft-4.0.0.jar:?]
Sep 22 09:41:42 s99ossv152 kafka[42245]: #011at
org.apache.kafka.raft.KafkaRaftClient.handleInboundMessage(KafkaRaftClient.java:2724)
~[kafka-raft-4.0.0.jar:?]
Sep 22 09:41:42 s99ossv152 kafka[42245]: #011at
org.apache.kafka.raft.KafkaRaftClient.poll(KafkaRaftClient.java:3460)
~[kafka-raft-4.0.0.jar:?]
Sep 22 09:41:42 s99ossv152 kafka[42245]: #011at
org.apache.kafka.raft.KafkaRaftClientDriver.doWork(KafkaRaftClientDriver.java:64)
[kafka-raft-4.0.0.jar:?]
Sep 22 09:41:42 s99ossv152 kafka[42245]: #011at
org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:136)
[kafka-server-common-4.0.0.jar:?]
Below metrics shows Fenceborker count as 2.0
kafka_controller_KafkaController_Value\{name="ActiveBrokerCount",} 1.0
kafka_controller_KafkaController_Value\{name="GlobalTopicCount",} 23.0
kafka_controller_KafkaController_Value{name="{*}FencedBrokerCount{*}",} 2.0
Please help us to resolve this issue.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
