The log indicates that the follower brokers can't connect to the leader broker. Does telnet from one broker to another work? You may also want to look at https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-OnEC2,whycan'tmyhigh-levelconsumersconnecttothebrokers ?
Thanks, Jun On Thu, Dec 26, 2013 at 6:52 PM, 陈小军 <[email protected]> wrote: > I have 4 kafka brokers, and create three topics, each topic has four > partions and 3 repilca. I use the kafka-trunk code. When I insert the data > to the kafka, suddenly 3 of brokers dead, and I check the logs, the error > is following > [2013-12-27 11:39:13,987] INFO Reconnect due to socket error: null > (kafka.consumer.SimpleConsumer) > gs,1] -> PartitionFetchInfo(3158155,200000000),[nelo2-crash-logs,3] > -> PartitionFetchInfo(15315,200000000) > (kafka.server.ReplicaFetcherThread) > java.net.ConnectException: źε2;9Hat sun.nio.ch.Net.connect0(Native Method) > > at sun.nio.ch.Net.connect(Net.java:465) > at sun.nio.ch.Net.connect(Net.java:457) > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:666) > at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57) > at kafka.consumer.SimpleConsumer.connect(SimpleConsumer.scala:44) > at kafka.consumer.SimpleConsumer.reconnect(SimpleConsumer.scala:57) > at > kafka.consumer.SimpleConsumer.liftedTree1$1(SimpleConsumer.scala:79) > at > kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(SimpleConsumer.scala:71) > at > kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(SimpleConsumer.scala:110) > at > kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:110) > at > kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:110) > at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33) > at > kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply$mcV$sp(SimpleConsumer.scala:109) > at > kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:109) > at > kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:109) > at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33) > at kafka.consumer.SimpleConsumer.fetch(SimpleConsumer.scala:108) > > > > 5149,2-9 88% > at kafka.consumer.SimpleConsumer.reconnect(SimpleConsumer.scala:57) > at > kafka.consumer.SimpleConsumer.liftedTree1$1(SimpleConsumer.scala:79) > at > kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(SimpleConsumer.scala:71) > at > kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(SimpleConsumer.scala:110) > at > kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:110) > at > kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:110) > kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33) > at > kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply$mcV$sp(SimpleConsumer.scala:109) > at > kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:109) > at > kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:109) > at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33) > at kafka.consumer.SimpleConsumer.fetch(SimpleConsumer.scala:108) > at > kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:96) > at > kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:88) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51) > [2013-12-27 11:39:13,986] WARN [ReplicaFetcherThread-1-3], Error in fetch > Name: FetchRequest; Version: 0; CorrelationId: 20401; ClientId: > ReplicaFetcherThread-1-3; ReplicaId: 2; MaxWait: 500 ms; MinBytes: 1 bytes; > RequestInfo: [nelo2-nor > mal-logs,1] -> > PartitionFetchInfo(3158155,200000000),[nelo2-crash-logs,3] -> > PartitionFetchInfo(15315,200000000) (kafka.server.ReplicaFetcherThread) > java.net.ConnectException: źε9;9Hat sun.nio.ch.Net.connect0(Native Method) > > at sun.nio.ch.Net.connect(Net.java:465) > at sun.nio.ch.Net.connect(Net.java:457) > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:666) > at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57) > at kafka.consumer.SimpleConsumer.connect(SimpleConsumer.scala:44) > at kafka.consumer.SimpleConsumer.reconnect(SimpleConsumer.scala:57) > at > kafka.consumer.SimpleConsumer.liftedTree1$1(SimpleConsumer.scala:79) > at > kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(SimpleConsumer.scala:71) > at > kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(SimpleConsumer.scala:110) > at > kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:110) > at > kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:110) > at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33) > at > kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply$mcV$sp(SimpleConsumer.scala:109) > at > kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:109) > at > kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:109) > at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33) > at kafka.consumer.SimpleConsumer.fetch(SimpleConsumer.scala:108) > at > kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:96) > at > kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:88) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51) > [2013-12-27 11:39:13,986] INFO Reconnect due to socket error: null > (kafka.consumer.SimpleConsumer) > [2013-12-27 11:39:13,987] INFO Reconnect due to socket error: null > (kafka.consumer.SimpleConsumer) > > Can someone alway meet this problem? and I don't why this problem is occur? > > > Best Regards > Jerry >
