Hi Team , First of all my warm regards to each and everyone in the community .
We are using kafka for centralizing the logging through our php application . Earlier our application was writing all the logs in a file in json line format , which we used to read using filebeat ( ELK family member ) and send it to logstash instace which in turn send it to kafka topics using kafka output plugin . Recently we enhanced our logger and made a producer which logs directly to kafka server using a kafka handler ( This handler uses phpkafka ( on the top of librdkafka) . We ran into situation where if broker was down the php application was not able to produce any more logs throwing error . The error which We recieved at consumer end was [2016-03-02 15:25:18,233] ERROR [ConsumerFetcherThread-console-consumer-14821_zk1-1456912451600-6bccbb09-0-1], Error for partition [cron,1] to broker 1:class kafka.common.NotLeaderForPartitionException (kafka.consumer.ConsumerFetcherThread) We have 3 zookeeper servers and 4 kafka brokers . I have topic with 5 partition and 3 replicas each . Now if 1 of the broker gets down , application should automatically detect leader and start writing in the available leader . But application ends with segementation fault . Please help . Regards Arvind Sharma Software Engg. Shopclues
