Anil created KAFKA-6281:
---------------------------
Summary: Kafka JavaAPI Producer failed with
NotLeaderForPartitionException
Key: KAFKA-6281
URL: https://issues.apache.org/jira/browse/KAFKA-6281
Project: Kafka
Issue Type: Bug
Reporter: Anil
Attachments: server1-controller.log, server2-controller.log
We are running Kafka (vesion kafka_2.11-0.10.1.0) in a 2 node cluster. We have
2 producers (Java API) acting on different topics. Each topic has single
partition. The topic where we had this issue, has one consumer running. This
set up has been running fine for 3 months, and we saw this issue. All the
suggested cases/solutions for this issue in other forums don't seem to apply
for my scenario.
Exception at producer;
{code}
-2017-11-25T17:40:33,035 [kafka-producer-network-thread | producer-1] ERROR
client.producer.BingLogProducerCallback - Encountered exception in sending
message ; > org.apache.kafka.common.errors.NotLeaderForPartitionException: This
server is not the leader for that topic-partition.
{code}
We haven't enabled retries for the messages, because this is transactional data
and we want to maintain the order.
Producer config:
{code}
bootstrap.servers : server1ip:9092
acks :all
retries : 0
linger.ms :0
buffer.memory :10240000
max.request.size :1024000
key.serializer : org.apache.kafka.common.serialization.StringSerializer
value.serializer : org.apache.kafka.common.serialization.StringSerializer
{code}
We are connecting to server1 at both producer and consumer. The controller log
at server2 indicates there is some shutdown happened at during sametime, but I
dont understand why this happened.
{color:red}[2017-11-25 17:31:44,776] DEBUG [Controller 2]: topics not in
preferred replica Map() (kafka.controller.KafkaController) [2017-11-25
17:31:44,776] TRACE [Controller 2]: leader imbalance ratio for broker 2 is
0.000000 (kafka.controller.KafkaController) [2017-11-25 17:31:44,776] DEBUG
[Controller 2]: topics not in preferred replica Map()
(kafka.controller.KafkaController) [2017-11-25 17:31:44,776] TRACE [Controller
2]: leader imbalance ratio for broker 1 is 0.000000
(kafka.controller.KafkaController) [2017-11-25 17:34:18,314] INFO
[SessionExpirationListener on 2], ZK expired; shut down all controller
components and try to re-elect
(kafka.controller.KafkaController$SessionExpirationListener) [2017-11-25
17:34:18,317] DEBUG [Controller 2]: Controller resigning, broker id 2
(kafka.controller.KafkaController) [2017-11-25 17:34:18,317] DEBUG [Controller
2]: De-registering IsrChangeNotificationListener
(kafka.controller.KafkaController) [2017-11-25 17:34:18,317] INFO
[delete-topics-thread-2], Shutting down
(kafka.controller.TopicDeletionManager$DeleteTopicsThread) [2017-11-25
17:34:18,317] INFO [delete-topics-thread-2], Stopped
(kafka.controller.TopicDeletionManager$DeleteTopicsThread) [2017-11-25
17:34:18,318] INFO [delete-topics-thread-2], Shutdown completed
(kafka.controller.TopicDeletionManager$DeleteTopicsThread) [2017-11-25
17:34:18,318] INFO [Partition state machine on Controller 2]: Stopped partition
state machine (kafka.controller.PartitionStateMachine) [2017-11-25
17:34:18,318] INFO [Replica state machine on controller 2]: Stopped replica
state machine (kafka.controller.ReplicaStateMachine) [2017-11-25 17:34:18,318]
INFO [Controller-2-to-broker-2-send-thread], Shutting down
(kafka.controller.RequestSendThread) [2017-11-25 17:34:18,318] INFO
[Controller-2-to-broker-2-send-thread], Stopped
(kafka.controller.RequestSendThread) [2017-11-25 17:34:18,319] INFO
[Controller-2-to-broker-2-send-thread], Shutdown completed
(kafka.controller.RequestSendThread) [2017-11-25 17:34:18,319] INFO
[Controller-2-to-broker-1-send-thread], Shutting down
(kafka.controller.RequestSendThread) [2017-11-25 17:34:18,319] INFO
[Controller-2-to-broker-1-send-thread], Stopped
(kafka.controller.RequestSendThread) [2017-11-25 17:34:18,319] INFO
[Controller-2-to-broker-1-send-thread], Shutdown completed
(kafka.controller.RequestSendThread) [2017-11-25 17:34:18,319] INFO [Controller
2]: Broker 2 resigned as the controller (kafka.controller.KafkaController)
[2017-11-25 17:34:18,353] DEBUG [IsrChangeNotificationListener] Fired!!!
(kafka.controller.IsrChangeNotificationListener) [2017-11-25 17:34:18,353]
DEBUG [IsrChangeNotificationListener] Fired!!!
(kafka.controller.IsrChangeNotificationListener) [2017-11-25 17:34:18,354] INFO
[BrokerChangeListener on Controller 2]: Broker change listener fired for path
/brokers/ids with children 1,2
(kafka.controller.ReplicaStateMachine$BrokerChangeListener) [2017-11-25
17:34:18,355] DEBUG [DeleteTopicsListener on 2]: Delete topics listener fired
for topics to be deleted
(kafka.controller.PartitionStateMachine$DeleteTopicsListener) [2017-11-25
17:34:18,362] INFO [AddPartitionsListener on 2]: Partition modification
triggered {"version":1,"partitions":{"0":[1]}} for path /brokers/topics/ESQ
(kafka.controller.PartitionStateMachine$PartitionModificationsListener)
[2017-11-25 17:34:18,368] INFO [AddPartitionsListener on 2]: Partition
modification triggered {"version":1,"partitions":{"0":[1]}} for path
/brokers/topics/Test1
(kafka.controller.PartitionStateMachine$PartitionModificationsListener)
[2017-11-25 17:34:18,369] INFO [AddPartitionsListener on 2]: Partition
modification triggered {"version":1,"partitions":{"0":[2]}} for path
/brokers/topics/ImageQ
(kafka.controller.PartitionStateMachine$PartitionModificationsListener)
[2017-11-25 17:34:18,374] INFO [AddPartitionsListener on 2]: Partition
modification triggered
{"version":1,"partitions":{"8":[1,2],"4":[1,2],"9":[2,1],"5":[2,1],"6":[1,2],"1":[2,1],"0":[1,2],"2":[1,2],"7":[2,1],"3":[2,1]}}
for path /brokers/topics/NMS_NotifyQ
(kafka.controller.PartitionStateMachine$PartitionModificationsListener)
[2017-11-25 17:34:18,375] INFO [AddPartitionsListener on 2]: Partition
modification triggered {"version":1,"partitions":{"0":[1]}} for path
/brokers/topics/TempBinLogReqQ @{color}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
