Your producer might get stuck if the kafka Broker becomes unreachable since there is no socketTimeout on new producer. We are adding that in KAFKA-2120.
Thanks, Mayuresh On Wed, Aug 5, 2015 at 3:47 PM, Chinmay Soman <chinmay.cere...@gmail.com> wrote: > After digging in the 0.8.2 code, it seems like the callback is not getting > invoked since 'handleDisconnections' is not adding a disconnected > ClientResponse to the list of responses. I do see a 'Node 0 disconnected' > message. However, I don't see a 'Cancelled request due to node being > disconnected' message. > > Is this expected ? Is any of this because I'm only using 1 Kafka broker > node ? Will the producer get stuck if the Kafka cluster is not reachable > for whatever reason ? > > On Wed, Aug 5, 2015 at 12:12 PM, Chinmay Soman <chinmay.cere...@gmail.com> > wrote: > > > Hey guys, > > > > We're using Kafka version 0.8.2.0 and using the Java producer > > (KafkaProducer) to write to a single Kafka broker with acks=1. There's a > > callback defined on every produce request which takes corrective action > if > > the write fails. > > > > What we see is that, if the broker is down for an extended period of > time, > > any write to the producer gets stuck, with this message: > > > > [2015-08-05 12:02:04,743] WARN Error in I/O with /........ > > (org.apache.kafka.common.network.Selector:276) > > java.net.ConnectException: Connection refused > > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) > > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) > > at org.apache.kafka.common.network.Selector.poll(Selector.java:238) > > at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:192) > > at > org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:191) > > at > org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:122) > > at java.lang.Thread.run(Thread.java:745) > > > > This goes on repeatedly and the callback is never invoked. Is there a way > > to get the Kafka EventThread to give up after a while ? > > > > > > -- > > Thanks and regards > > > > Chinmay Soman > > > > > > -- > Thanks and regards > > Chinmay Soman > -- -Regards, Mayuresh R. Gharat (862) 250-7125
