It’s not totally clear, but this may be https://issues.apache.org/jira/plugins/servlet/mobile#issue/KAFKA-7941
For which there is a fix that is very nearly approved: https://github.com/apache/kafka/pull/6283 Paul > On Jun 5, 2019, at 1:26 AM, Srinivas, Kaushik (Nokia - IN/Bangalore) > <kaushik.srini...@nokia.com> wrote: > > Hello, > Anyone has any information on this issue. > Created a critical ticket for the same, since this is a major stability issue > for connect framework. > https://issues.apache.org/jira/browse/KAFKA-8485?filter=-2 > > Thanks. > Kaushik, > NOKIA > > From: Srinivas, Kaushik (Nokia - IN/Bangalore) > Sent: Monday, June 03, 2019 5:22 PM > To: dev@kafka.apache.org > Cc: Basil Brito, Aldan (Nokia - IN/Bangalore) <aldan.basil_br...@nokia.com> > Subject: kafka connect stops consuming data when kafka broker goes down > > Hello kafka dev, > > We are encountering an issue when kafka connect is running hdfs sink > connector pulling data from kafka and writing to hdfs location. > In between when the data is flowing in the pipeline from producer -> kafka > topic -> kafka connect hdfs sink connector -> hdfs, > If even one of the kafka broker goes down, the connect framework stops > responding. Stops consuming records and REST API also becomes not interactive. > > Until the kafka connect framework is restarted, it would not pull the data > from kafka and REST api remains inactive. Nothing is coming in the logs as > well. > Checked the topics in kafka used by connect, everything has been reassigned > to another broker and has the leader available. > > Has anyone encountered this issue ? what would be the expected behavior ? > > Thanks in advance > Kaushik
