little brother ma created KAFKA-7238:
----------------------------------------
Summary: Producer can't update Metadata when two brokers break
down and restart the first one
Key: KAFKA-7238
URL: https://issues.apache.org/jira/browse/KAFKA-7238
Project: Kafka
Issue Type: Bug
Components: clients
Affects Versions: 0.11.0.3, 0.11.0.1
Environment: os: 14.04.1-Ubuntu
java: 1.8.0_151-b12
Reporter: little brother ma
There is a kafka cluster with two brokers( broker ids are 70 and 67),I create a
topic named topic2 with 2 partitions and 1 replica, and partition 0 is on the
broker 70,and partition 1 is on the broker 67.
While sending datas， I stop broker 70 first, and wait 5 ms， stop broker 67 .
And then ,I restart broker 70, the producer client can't update metadata
and don't switch to connet broker 70，otherwise， it always try to connect
broker 67.
The log before stop the broker:
================send
The message size: 12, key: null, partition:0, offset:29
[2018-08-02 19:59:10,180] DEBUG Sending metadata request (type=MetadataRequest,
topics=topic2) to node 10.93.132.67:9092 (id: 67 rack: null)
(org.apache.kafka.clients.NetworkClient)
[2018-08-02 19:59:10,184] DEBUG Updated cluster metadata version 11 to
Cluster(id = 3fL1MXQtRo6Ujmxa9dS3eA, nodes = [10.93.132.70:9092 (id: 70 rack:
null), 10.93.132.67:9092 (id: 67 rack: null)], partitions = [Partition(topic =
topic2, partition = 1, leader = 70, replicas = [70], isr = [70]),
Partition(topic = topic2, partition = 0, leader = 67, replicas = [67], isr =
[67])]) (org.apache.kafka.clients.Metadata)
================send
The message size: 12, key: null, partition:1, offset:69
================send
The message size: 12, key: null, partition:0, offset:30
================send
The message size: 12, key: null, partition:1, offset:70
================send
The message size: 12, key: null, partition:0, offset:31
================send
The message size: 12, key: null, partition:1, offset:71
================send
The message size: 12, key: null, partition:0, offset:32
The log after stop the broker 70 :
[2018-08-02 20:02:50,049] DEBUG Node 70 disconnected.
(org.apache.kafka.clients.NetworkClient)
[2018-08-02 20:02:50,049] WARN Connection to node 70 could not be established.
Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2018-08-02 20:02:50,049] DEBUG Give up sending metadata request since no node
is available (org.apache.kafka.clients.NetworkClient)
[2018-08-02 20:02:50,100] DEBUG Give up sending metadata request since no node
is available (org.apache.kafka.clients.NetworkClient)
[2018-08-02 20:02:50,151] DEBUG Initialize connection to node 10.93.132.70:9092
(id: 70 rack: null) for sending metadata request
(org.apache.kafka.clients.NetworkClient)
[2018-08-02 20:02:50,151] DEBUG Initiating connection to node 10.93.132.70:9092
(id: 70 rack: null) (org.apache.kafka.clients.NetworkClient)
================send
The message size: 12, key: null, partition:0, offset:95
[2018-08-02 20:02:51,156] DEBUG Connection with /10.93.132.70 disconnected
(org.apache.kafka.common.network.Selector)
java.net.ConnectException: Connection refused: no further information
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at
org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at
org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:95)
at
org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:361)
at org.apache.kafka.common.network.Selector.poll(Selector.java:326)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:433)
at
org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.run(KafkaAdminClient.java:954)
at java.lang.Thread.run(Thread.java:745)
[2018-08-02 20:02:51,157] DEBUG Node 70 disconnected.
(org.apache.kafka.clients.NetworkClient)
[2018-08-02 20:02:51,157] WARN Connection to node 70 could not be established.
Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2018-08-02 20:02:51,157] DEBUG Give up sending metadata request since no node
is available (org.apache.kafka.clients.NetworkClient)
[2018-08-02 20:02:51,208] DEBUG Initialize connection to node 10.93.132.70:9092
(id: 70 rack: null) for sending metadata request
(org.apache.kafka.clients.NetworkClient)
[2018-08-02 20:02:51,208] DEBUG Initiating connection to node 10.93.132.70:9092
(id: 70 rack: null) (org.apache.kafka.clients.NetworkClient)
[2018-08-02 20:02:52,216] DEBUG Connection with /10.93.132.70 disconnected
(org.apache.kafka.common.network.Selector)
java.net.ConnectException: Connection refused: no further information
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at
org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at
org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:95)
at
org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:361)
at org.apache.kafka.common.network.Selector.poll(Selector.java:326)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:433)
at
org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.run(KafkaAdminClient.java:954)
at java.lang.Thread.run(Thread.java:745)
[2018-08-02 20:02:52,216] DEBUG Node 70 disconnected.
(org.apache.kafka.clients.NetworkClient)
[2018-08-02 20:02:52,216] WARN Connection to node 70 could not be established.
Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2018-08-02 20:02:52,216] DEBUG Give up sending metadata request since no node
is available (org.apache.kafka.clients.NetworkClient)
[2018-08-02 20:02:52,267] DEBUG Give up sending metadata request since no node
is available (org.apache.kafka.clients.NetworkClient)
[2018-08-02 20:02:52,318] DEBUG Initialize connection to node 10.93.132.70:9092
(id: 70 rack: null) for sending metadata request
(org.apache.kafka.clients.NetworkClient)
[2018-08-02 20:02:52,318] DEBUG Initiating connection to node 10.93.132.70:9092
(id: 70 rack: null) (org.apache.kafka.clients.NetworkClient)
================send
The message size: 12, key: null, partition:0, offset:96
The broker 67 is working!
The log after stop the broker 70 ,broker 67 ,and restart broker 70 :
[2018-08-02 20:05:15,136] DEBUG Node 67 disconnected.
(org.apache.kafka.clients.NetworkClient)
[2018-08-02 20:05:15,136] WARN Connection to node 67 could not be established.
Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2018-08-02 20:05:15,137] DEBUG Give up sending metadata request since no node
is available (org.apache.kafka.clients.NetworkClient)
[2018-08-02 20:05:15,177] DEBUG Initiating connection to node 10.93.132.67:9092
(id: 67 rack: null) (org.apache.kafka.clients.NetworkClient)
[2018-08-02 20:05:16,183] DEBUG Connection with /10.93.132.67 disconnected
(org.apache.kafka.common.network.Selector)
java.net.ConnectException: Connection refused: no further information
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at
org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at
org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:95)
at
org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:361)
at org.apache.kafka.common.network.Selector.poll(Selector.java:326)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:433)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:224)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:162)
at java.lang.Thread.run(Thread.java:745)
[2018-08-02 20:05:16,183] DEBUG Node 67 disconnected.
(org.apache.kafka.clients.NetworkClient)
[2018-08-02 20:05:16,183] WARN Connection to node 67 could not be established.
Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2018-08-02 20:05:16,183] DEBUG Give up sending metadata request since no node
is available (org.apache.kafka.clients.NetworkClient)
[2018-08-02 20:05:16,233] DEBUG Give up sending metadata request since no node
is available (org.apache.kafka.clients.NetworkClient)
[2018-08-02 20:05:16,242] DEBUG Initiating connection to node 10.93.132.67:9092
(id: 67 rack: null) (org.apache.kafka.clients.NetworkClient)
[2018-08-02 20:05:17,247] DEBUG Connection with /10.93.132.67 disconnected
(org.apache.kafka.common.network.Selector)
java.net.ConnectException: Connection refused: no further information
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at
org.apache.kafka.common.network.PlaintextTransportLayer.finishConnect(PlaintextTransportLayer.java:50)
at
org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:95)
at
org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:361)
at org.apache.kafka.common.network.Selector.poll(Selector.java:326)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:433)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:224)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:162)
at java.lang.Thread.run(Thread.java:745)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
